Azure Data Engineer

Azure Data Engineer

Location: Branchville, NJ, Charlotte, NC and Hartford CT Years of Experience: 7-10 Years

Job Summary: We are seeking a highly skilled Data Engineer with a strong background in Azure Databricks to join our dynamic team. The ideal candidate will have 7-10 years of experience in data engineering, with a proven track record of designing, building, and maintaining scalable data pipelines and data solutions. This role requires a deep understanding of data architecture, data modeling, and ETL processes, with a specific emphasis on leveraging Azure Databricks for data processing and analytics.

Responsibilities:

• Design, develop, and maintain robust data pipelines using Azure Databricks to support data ingestion, transformation, and storage.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high quality data solutions.
• Implement best practices for data governance, data quality, and data security within the Azure environment.
• Optimize existing data workflows and processes for performance and scalability.
• Monitor and troubleshoot data pipeline performance issues, ensuring timely resolution.
• Document data engineering processes, architecture, and workflows for knowledge sharing and compliance.
• Stay updated with the latest trends and technologies in data engineering and Azure Databricks.

Mandatory Skills:

• Strong expertise in Azure Databricks, including experience with Spark, Delta Lake, and data processing frameworks.
• Proficient in programming languages such as Python, Scala, or SQL for data manipulation and analysis.
• Experience with data modeling, ETL processes, and data warehousing concepts.
• Solid understanding of cloud data architecture and services, particularly within the Azure ecosystem.
• Familiarity with data visualization tools and techniques.

Preferred Skills:

• Experience with Azure Data Factory, Azure Synapse Analytics, and other Azure data services.
• Knowledge of machine learning concepts and frameworks.
• Experience with CI/CD practices and tools for data engineering.
• Familiarity with containerization technologies such as Docker and Kubernetes.
• Strong analytical and problem solving skills, with the ability to work independently and in a team environment.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• 7-10 years of experience in data engineering or a related field.
• Proven experience working with Azure Databricks and related technologies.
• Strong communication skills, with the ability to convey complex technical concepts to non technical stakeholders.

If you are a passionate Data Engineer with a strong background in Azure Databricks and are looking for an exciting opportunity to make an impact, we encourage you to apply!