Data Engineer

Data EngineerWe are seeking a highly skilled Data Engineer with strong expertise in SQL, Python, Apache Spark, and hands-on experience with Azure cloud services, including Azure Databricks and Azure Synapse Analytics. The ideal candidate will also have exposure to or interest in AI/ML integration within data engineering pipelines. You will play a key role in building and optimizing data architecture, enabling advanced analytics and machine learning capabilities.Key Responsibilities:Design, build, and maintain scalable and efficient data pipelines for large-scale data processing.Develop and optimize data workflows using Apache Spark, SQL, and Python.Implement and manage cloud-based data solutions on Azure, including Databricks and Synapse Analytics.Integrate data from multiple sources, ensuring data quality, consistency, and governance.Collaborate with data scientists, analysts, and business teams to support AI/ML initiatives.Monitor and improve data pipeline performance and troubleshoot production issues.Implement best practices in data architecture, security, and compliance.Required Skills:Strong experience with SQL.Proficiency in Python for data engineering tasks and automation.Expertise in Apache Spark for distributed data processing.Hands-on experience with Azure Databricks and Azure Synapse Analytics.Understanding of data lake, data warehouse, and ETL/ELT concepts.Familiarity with integrating AI/ML models into data pipelines is a plus.Experience with CI/CD for data pipelines is a bonus.