Sr Data Quality Engineer - Azure Focus

Sr Data Quality Engineer - Azure Focus
Chicago, IL (loop/downtown) - 2 days work from home (WFH), 3 days onsite

Summary:
This position focuses on building automated tesing frameworks to ensure data quality and reliability across data pipelines in our Azure, Databricks, SQL Server/Azure SQL and Medallion Architecture (Bronze, Silver, Gold layers) environment.


We use Python and SQL extensively on our data engineering team, and are looking for candidates who can come to the table with these skills, and a candidate coming from an Azure/Databricks environment.


We are also looking for candidates that have strong experience building automated validation frameworks using tools - if you have experience with DQX (Databricks Data Quality Framework), that would be awesome, but anything similar works.


Last, we are leaning into AI on our engineering teams, and you'll have the opportunity to leverage tools like Claude Code, so any experience in this area is also a nice to have.


What you'll be doing

• Design and implement scalable end-to-end testing frameworks for data pipelines
• Validate ingestion from APIs, SQL Server, and flat files
• Ensure data quality across Medallion architecture layers (Bronze, Silver, Gold)
• Build automated checks for schema validation, data integrity, and transformations
• Develop reusable validation patterns using Great Expectations or similar frameworks
• Leverage Claude Code and prompt engineering to accelerate test generation and standardization
• Create reusable AI-driven testing assets (.md skills, templates) and workflows for AI-assisted coding and testing
• Integrate testing into CI/CD pipelines (Azure DevOps, GitHub Actions)
• Collaborate with data engineers, Product, and DevOps to define data quality standards and acceptance criteria
• Monitor and improve data reliability, observability, and test coverage
• Investigate data quality issues and drive root-cause resolution

Skills we're seeking

• 5+ years of experience with Data Engineering and/or Data Quality Engineering
• Must have strong experience with Data Quality Engineering
  • Must have experience with data validation tools such as DQX, Great Expectations, etc.
• Must have strong experience with Data (Quality) Engineering in Azure environments
• Must have experience with Medallion Architecture
• Must have strong experience with SQL Server
• Python experience

Nice to haves

• Bachelor's or Master's Degree in an IT related field
• Spark and/or PySpark experience
• Experience with Claud Code
• Experience with Azure DevOps, CI/CD pipeliens, etc.
  • Bicep experience would be awesome, or any IaC for that matter