Lead Data Quality Engineer

Role Overview

We are looking for a Senior Data Quality Lead to establish and own the data quality practice for our enterprise data platform built on Microsoft Fabric. You will be the founding voice for data quality — shaping the standards, validation patterns, and certification processes that ensure trusted, audit-ready data flows from source ingestion through to curated analytical layers. This is a high-impact, greenfield opportunity to build discipline from the ground up within a fast-moving private credit lending environment.

What You'll Own

• Data Quality Framework — Architect the enterprise data quality framework — including validation rules, acceptance thresholds, exception handling workflows, and escalation paths — spanning the full medallion architecture (Bronze, Silver, Gold).

• Certification & Reconciliation — Design and deliver source-to-target reconciliation packs and regression test suites that serve as the evidence base for production certification of curated datasets.

• Governance Integration — Integrate quality rules, certification status, and data lineage into Microsoft Purview to ensure a unified governance experience across the enterprise catalog.

• Data Quality Observability — Establish an observability layer that provides continuous visibility into data health, rule outcomes, reconciliation status, and certification readiness — delivered through a reporting mechanism best fits the framework design.

Responsibilities

• Define the data quality rule taxonomy — categorizing validations by type (completeness, accuracy, consistency, timeliness, uniqueness), severity, and remediation path — and maintain it as a living standard that evolves with the platform.
• Establish Bronze-layer entry criteria grounded in the organization's data asset register, including schema conformance checks, null-rate thresholds, row-count validations, and source-system freshness expectations.
• Design Silver-layer curation rules that validate transformation logic, enforce referential integrity across related datasets, and flag records that fail business-rule conformance before they reach downstream consumers.
• Define Gold-layer certification criteria and production-readiness signoff processes in partnership with data engineering and business stakeholders; own the evidence packs that support each certification decision.
• Build automated reconciliation workflows against core source systems — including loan servicing and accounting platforms (ACBS/CLS), general ledger (NetSuite), portfolio management (Black Mountain), and Deal pipeline (Deal Cloud) etc.
• Design the observability and reporting strategy for data quality — defining what metrics to track (rule pass/fail rates, exception volumes, time-to-resolution, reconciliation variance trends), how they are surfaced, and who receives them at each level of the organization.
• Establish exception management workflows including triage criteria, ownership assignment, remediation SLAs, and feedback loops that convert recurring defects into preventive rule updates.
• Conduct root-cause analysis for systemic data defects, document findings and remediation playbooks, and ensure institutional knowledge is retained through well-maintained framework documentation.

Required Skills

• DQ Framework Design: Proven track record of standing up a data quality practice or framework — defining rule taxonomies, acceptance thresholds, certification workflows, and observability strategies in a data platform environment.
• Microsoft Fabric: Lakehouse, Warehouse, Data Pipelines, Notebooks, OneLake; working knowledge of Delta Lake and Parquet formats.
• Microsoft Purview: Data cataloging, classification, lineage tracking, and sensitivity labeling integrated with quality workflows.
• SQL: Advanced proficiency in writing reconciliation queries, profiling logic, anomaly detection, and performance tuning.
• Data Quality Reporting & Observability: Ability to design data quality observability solutions — defining KPIs, alerting thresholds, and reporting cadences that give stakeholders confidence in data health across the platform.
• Financial Data Systems: Hands-on reconciliation experience against loan accounting, GL, or portfolio systems in lending or financial services environments.

Preferred Experience

• DQ Framework Design : Define standards, implementing automated validation, and designing pipelines that ensure accuracy and completeness
• Observability Tools: Experience with data observability or data quality platforms such as Great Expectations, Soda, Deequ, Monte Carlo, or custom-built validation frameworks.
• Audit and Controls: Exposure to SOX-type controls, audit processes, regulatory reporting frameworks, control evidence, and exception sign-off workflows.
• Metadata and Lineage: Experience contributing to enterprise metadata, business glossary, standardized KPI definitions, lineage documentation, and impact analysis for upstream changes.

Soft Skills

• Agile Mindset: Ability to iterate quickly, pivot based on stakeholder feedback, and prioritize controls based on risk and business impact.
• Communication: Strong ability to explain data quality findings to engineers, analysts, Product Managers, auditors, and senior leadership in business terms.
• Problem Solving: A proactive approach to identifying data risks, isolating root causes, and creating durable controls before issues reach production reporting.
• Attention to Detail: Comfort operating in high-accuracy financial data environments where small variances can have material downstream impact.