Azure Data Engineer

Azure Data EngineerLocation: Branchville, NJ, Charlotte, NC and Hartford CTYears of Experience: 7-10 YearsJob Summary: We are seeking a highly skilled Data Engineer with a strong background in Azure Databricks to join our dynamic team. The ideal candidate will have 7-10 years of experience in data engineering, with a proven track record of designing, building, and maintaining scalable data pipelines and data solutions. This role requires a deep understanding of data architecture, data modeling, and ETL processes, with a specific emphasis on leveraging Azure Databricks for data processing and analytics.Responsibilities:Design, develop, and maintain robust data pipelines using Azure Databricks to support data ingestion, transformation, and storage.Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high quality data solutions.Implement best practices for data governance, data quality, and data security within the Azure environment.Optimize existing data workflows and processes for performance and scalability.Monitor and troubleshoot data pipeline performance issues, ensuring timely resolution.Document data engineering processes, architecture, and workflows for knowledge sharing and compliance.Stay updated with the latest trends and technologies in data engineering and Azure Databricks.Mandatory Skills:Strong expertise in Azure Databricks, including experience with Spark, Delta Lake, and data processing frameworks.Proficient in programming languages such as Python, Scala, or SQL for data manipulation and analysis.Experience with data modeling, ETL processes, and data warehousing concepts.Solid understanding of cloud data architecture and services, particularly within the Azure ecosystem.Familiarity with data visualization tools and techniques.Preferred Skills:Experience with Azure Data Factory, Azure Synapse Analytics, and other Azure data services.Knowledge of machine learning concepts and frameworks.Experience with CI/CD practices and tools for data engineering.Familiarity with containerization technologies such as Docker and Kubernetes.Strong analytical and problem solving skills, with the ability to work independently and in a team environment.Qualifications:Bachelor's degree in Computer Science, Information Technology, or a related field.7-10 years of experience in data engineering or a related field.Proven experience working with Azure Databricks and related technologies.Strong communication skills, with the ability to convey complex technical concepts to non technical stakeholders.