Data Engineer and Platform Manager - 27739

Data Engineering & Platform Manager - Azure Databricks and Medallion Architecture

Impact Business Group is recruiting a Data Engineering & Platform Manager for a manufacturing client. You'll take hands-on ownership of its enterprise Azure Databricks platform. Your first assignment is to assess the legacy environment and the newer implementation, understand how teams use both, and lead the work to make the newer platform meet the business's reporting, analytics, and AI needs.

You'll report to the VP of Data, Analytics & AI and work together to set the technical roadmap. This is an individual contributor role with no direct reports. You'll make architecture decisions and build, test, and implement changes yourself.

What you'll do

Trace critical data flows from source systems to downstream reports. Compare the two environments, decide what to keep or rework, and validate changes before moving business-critical workloads.

Define and implement the medallion architecture. Set clear standards for bronze, silver, and gold layers, and build and maintain production pipelines from ERP, CRM, and other enterprise systems.

Own Databricks workspace administration, compute configuration, and pipeline operations. Monitor reliability, performance, and spend, and resolve production issues.

Strengthen security and governance with Unity Catalog. Implement access controls, track data lineage and ownership, and establish data quality checks and monitoring.

Work with Finance, Supply Chain, and analytics teams to reconcile data and define what must pass before a workload moves. Document pipeline standards and operational runbooks

What you'll bring

7-12 years of data engineering experience, including 3+ years of hands-on enterprise Databricks administration and pipeline development.

Production experience with Azure Databricks, Delta Lake, Unity Catalog, medallion architecture, and pipeline orchestration using Databricks Workflows, Azure Data Factory, or equivalent tools.

Strong SQL and Python skills, proficiency with Azure data services, and experience integrating ERP data into an enterprise platform.

Experience implementing data access controls, lineage, and data quality standards, and translating business requirements into architecture and implementation decisions.

Bachelor's degree in computer science, data engineering, information systems, or a related field.

Preferred experience

PySpark, SAP, Microsoft Dynamics GP, Databricks compute and cost optimization, or data engineering in regulated manufacturing.

Travel up to 15% as needed.