Senior Data Engineer

Senior Data Engineer

We are looking for a Senior Data Engineer with a strong foundation in modern data engineering, DevOps, and cloud data platforms. The primary focus of this role is designing, building, and operating scalable, reliable data pipelines and data products that support analytics, reporting, and AI initiatives across the organization.

In addition to building and operating reliable data pipelines, this role will contribute to automation, data quality, metadata management, and governance workflows. The successful candidate will bring strong data engineering expertise and an interest in expanding their knowledge of data governance and AI-enabled automation as these capabilities continue to evolve.

While our current platform is centered on Azure and Microsoft Fabric, we welcome candidates with strong data engineering experience on AWS, Databricks, or other modern data platforms, provided their skills are transferable and they demonstrate strong data engineering fundamentals.

This role is for a senior data engineer who enjoys building modern data platforms, scalable data pipelines, and trusted data products. The primary focus will be delivering high-quality data engineering solutions that support business, analytics, and AI needs. The individual will also have opportunities to contribute to governance patterns, platform automation, and controls embedded within CI/CD processes while developing these skills as the organization's capabilities mature.

Data Engineering FoundationsData Governance & Platform StewardshipAI & Automation EnablementDevOps & Platform EnablementDesign, build, and operate batch and streaming data pipelines on modern cloud data platforms.Develop robust ETL/ELT processes using SQL, Python, and PySpark with strong error handling, monitoring, and cost awareness.Implement layered / medallion data architectures and analytics-ready data models to support BI and AI workloads.Partner with analysts and data scientists to deliver trusted, production-grade data assets.

Data & AI Governance (Primary Focus)

Partner with platform and business stakeholders to support and enhance data governance practices using Microsoft Purview or comparable governance platforms.Contribute to governance standards and best practices for data products, analytics solutions, AI/ML features, and agent-based workflows as organizational capabilities continue to evolve.Help incorporate governance considerations into platform workflows and data engineering practices rather than treating governance as an after-the-fact process.Support ongoing data and AI governance initiatives and grow into greater ownership of governance patterns as experience and organizational standards evolve.

AI Agents & Automation

Contribute to the design and implementation of AI-enabled tools and automation workflows that support data platform and governance operations.Partner with technical teams to explore and develop automation that assists with metadata enrichment, data quality checks, anomaly detection, and governance workflows.Help ensure AI-enabled tools and automation operate only on approved, governed data sources with appropriate logging and auditability.Build knowledge of evolving AI governance patterns, including agent lifecycle management and input/output traceability, with support from the broader team.

DevOps & Platform Enablement

Implement and maintain CI/CD pipelines for data and AI assets.Promote infrastructure-as-code practices (Terraform, Bicep, or equivalent) for repeatable, governed environments.Define environment promotion paths (dev, test, prod) with embedded governance and policy checks.

Reliable, scalable data pipelines and data products support business reporting, analytics, and AI initiatives.Data assets are discoverable, trusted, and auditable, with governance considerations incorporated into the platform.The individual contributes to safe automation of platform and governance workflows while continuing to build AI and governance expertise.Teams scale analytics and AI capabilities while maintaining data quality, security, and operational efficiency.

Required Qualifications

• Bachelor's degree AND 5+ years of experience in data engineering or data platform engineering in a production environment.
• SQL and hands-on experience with Python and PySpark.
• Experience in data engineering on at least one major cloud or data platform (Azure, AWS, Databricks, or equivalent), with skills that are transferable across platforms.
• Experience with Git-based development and CI/CD practices.
• Working knowledge of data governance concepts such as cataloging, lineage, data quality, or access control.
• Demonstrated engineering judgment and the ability to balance delivery speed, reliability, safety, and governance considerations.

Preferred Qualifications

• Exposure to Microsoft Purview or comparable data governance platforms. Interest in or experience supporting AI agents, LLM-based automation, intelligent assistants, or other automation for operational or platform workflows. Exposure to embedding governance controls into DevOps workflows or an interest in developing a policy-as-code mindset.
• Familiarity with Microsoft Fabric or Databricks-centric architectures, and the ability to learn new platforms quickly.