Senior Data Engineer (Azure Databricks)

Senior Data Engineer (Azure Databricks) Location: Houston, TX (Onsite) Duration: 6 10 Months Contract Interview: Virtual Technical Skills: Azure Databricks | Delta Lake | PySpark | SQL | Scala | Databricks Workflows | Databricks Jobs | Unity Catalog | Azure DevOps | CI/CD | YAML Pipelines | Data Migration | ETL/ELT | Data Governance | Data Lineage | Alteryx Conversion | Azure Data Platform Note: Must be local with DL copy LinkedIn with location and photo (before 2023) Position Overview Our client is seeking a Senior Data Engineer with deep hands-on expertise in Azure Databricks to design, develop, and manage enterprise-scale data platforms. The ideal candidate will have strong experience with Databricks Workflows, Delta Lake, PySpark, Unity Catalog, Azure DevOps, and CI/CD automation . This role will focus on building code-driven data solutions, modernizing legacy data platforms, and migrating existing workflows into scalable Databricks-based architectures. Required Skills & Experience Strong hands-on experience with Azure Databricks engineering and development . Expertise in building data pipelines using: PySpark SQL Scala Delta Lake Experience with Databricks Workflows and Jobs for production orchestration. Strong knowledge of Databricks Unity Catalog including governance, security, and lineage. Experience with Azure DevOps, CI/CD pipelines, YAML, and deployment automation . Experience migrating legacy data solutions into modern cloud data platforms. Strong understanding of enterprise data architecture and data engineering best practices. Others Qualifications / Certifications Databricks certifications highly preferred: Databricks Certified Data Engineer Associate Databricks Certified Data Engineer Professional Microsoft certifications preferred: Azure Data Engineer Associate (DP-203) Azure Fundamentals (AZ-900) or equivalent Azure knowledge Experience converting Alteryx workflows into PySpark/SQL solutions is a major plus. Familiarity with legacy ETL modernization and reverse-engineering existing workflows. Key Responsibilities Design, develop, and optimize end-to-end data pipelines using Azure Databricks, Delta Lake, PySpark, SQL, and Scala . Build and manage production workflows using Databricks Workflows and Jobs for pipeline orchestration and automation. Lead migration efforts from legacy data platforms into modern Databricks environments. Convert legacy ETL workflows (including Alteryx workflows) into scalable PySpark/SQL-based solutions . Implement data governance, security, access controls, schemas, and lineage using Databricks Unity Catalog . Develop and maintain CI/CD processes for Databricks deployments using Azure DevOps, YAML pipelines, and automation frameworks . Optimize data processing performance, reliability, and scalability across enterprise data platforms. #J-18808-Ljbffr Akaasa Technologies