Azure Data Engineer

Lead Azure Data Engineer


New York, NY 10017


Contract - Long Term


Azure Cloud Expertise:

• Design, implement, and optimize data solutions on the Azure cloud platform, ensuring scalability, security, and performance.

Azure Data Factory and Databricks:

• Develop and maintain data pipelines using Azure Data Factory for efficient and reliable data movement.
• Leverage Azure Databricks for analytics and data exploration

SQL and DBT:

• Design and optimize SQL-based queries for data extraction, transformation, and loading processes.
• Implement and manage DBT (Data Build Tool) workflows to transform raw data into meaningful insights, ensuring data quality and consistency.

Data Architecture and Modeling:

• Collaborate with data architects to design and implement scalable and maintainable data models.
• Ensure adherence to data governance and compliance standards in data modeling and architecture.

Performance Optimization:

• Identify and address performance bottlenecks in data pipelines and database queries.
• Implement strategies to optimize data processing and reduce latency in data delivery.

Collaboration and Documentation:

• Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver solutions.
• Document data engineering processes, data flows, and architecture to facilitate knowledge transfer and ensure maintainability.

Qualifications:

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Data Engineer with a focus on Azure cloud technologies.
• Strong proficiency in Azure Data Factory, Azure Databricks, SQL, and DBT.
• 10+ years in the data and analytics space with experience in designing and implementing scalable and robust data architectures.
• Excellent problem-solving skills and the ability to troubleshoot complex data issues.
• Strong communication skills and the ability to collaborate effectively with cross-functional teams.
• Relevant certifications in Azure cloud technologies are a plus.
• Nice to have:

• • Python
  • (Re)Insurance industry experience