Azure Senior Data Lead

Maximum Salary (US): 148000Minimum Salary (US): 100000Other Requirements Primary Skill: Azure Databricks / Data EngineeringSecondary Skills: PySpark, SQL, Python, Azure Data Factory, Delta Lake, ADLS Gen2Job SummaryJob DescriptionWe are looking for an experienced Databricks Data Engineer to design, develop, and maintain scalable data engineering solutions on the Azure cloud platform. The candidate should have strong hands-on experience with Azure Databricks, Apache Spark, PySpark, SQL, Delta Lake, and Azure Data Factory (ADF), with a solid understanding of modern data lakehouse architectures.Key ResponsibilitiesDesign and develop scalable data pipelines using Azure Databricks, PySpark, Spark SQL, and Python.Build and maintain ETL/ELT pipelines for batch and large-scale data processing.Develop data ingestion and transformation workflows using Azure Data Factory (ADF).Design and implement Delta Lake / Lakehouse solutions.Work with Azure Data Lake Storage Gen2 (ADLS Gen2) for data ingestion, storage, and processing.Develop optimized PySpark and SQL transformations for large datasets.Implement Delta tables, schema management, data quality checks, and performance optimization.Work with Databricks Workflows/Jobs for pipeline orchestration and scheduling.Implement data governance and access controls using Unity Catalog.Monitor pipelines, troubleshoot failures, and optimize Spark workloads.Implement CI/CD processes using Azure DevOps and Git.Collaborate with Data Architects, Data Analysts, BI teams, and business stakeholders.Support downstream reporting and analytics solutions such as Power BI.Key ResponsibilitiesGood to HaveDatabricks SQLStructured StreamingKafka or Event HubsPower BITerraformCI/CD automationData modelingData quality frameworksMicrosoft Fabric exposureSkill RequirementsExperience9+ years of overall Data Engineering experience.4+years of hands-on Databricks/Spark experience preferred.Experience delivering cloud-based data platforms on Microsoft Azure.Other RequirementsPrimary Skill: Azure Databricks / Data EngineeringSecondary Skills: PySpark, SQL, Python, Azure Data Factory, Delta Lake, ADLS Gen2