Senior Data Engineer (Databricks)

A leading infrastructure and transportation group is seeking a hands-on Senior Data Engineer to architect and build our next-generation Enterprise Reporting & Data Warehouse (ERDW.Next) on Databricks. Acting as the hands-on technical owner, you will drive target-state design, medallion architecture (Delta Lake), and legacy migrations off SQL Server/Synapse while remaining actively in the codebase. You will partner with our Enterprise Architect, Power BI team, business users, and offshore delivery partners across finance, construction, and safety domains.QUALIFIED CANDIDATES SHOULD HAVE EXPERIENCE WITH:Serving as a Databricks SMEERP integrations (strong plus)Python / PySparkPerformance tuning and optimizationDatabricks maintenance, including VACUUMData governance and best practicesSemantic modelingRESPONSIBILITIESArchitecture & Strategy: Design the ERDW.Next lakehouse (medallion/gold layer star schema) and govern data modeling, batch/streaming, and Delta Live Tables standards.Build & Migration: Write production-grade PySpark, Spark SQL, and Delta Live Tables pipelines orchestrated via Databricks Workflows. Re-engineer legacy SSIS/T-SQL/Synapse logic and ingest ERP data (JD Edwards, Anaplan) using CDC and Auto Loader.Platform Operations: Optimize cost/performance using Photon, Liquid Clustering, and OPTIMIZE. Manage CI/CD pipelines via Git and Databricks Asset Bundles while maintaining strict SLAs.Governance & AI: Implement Unity Catalog governance (entitlements, lineage, audit) and curate Databricks Genie Agents for natural-language analytics.Partner & Legacy Management: Support legacy platforms during transition and maintain code quality standards across offshore partner teams. Leverage agentic AI coding tools safely to accelerate delivery.