Senior Data Engineer

Job Title: : Data Engineer Lead

Location: Lisle, Illinois

Day 1 Onsite at customer location

Responsibilities

• Build trusted data products and pipelines. Design, develop and maintain reliable, reusable data products and pipelines that support service product creation, sales, contract management, billing, claims, downstream execution and Performance Management.

• Discover and translate business processes into data requirements. Profile unfamiliar datasets; map workflows and system handoffs; identify grain, keys, relationships and temporal behavior; clarify business rules; investigate exceptions; and define testable requirements and acceptance criteria with product and business partners.

• Establish domain and performance data models. Define consistent entities, relationships, identifiers and lifecycle states for service products, policies, eligibility, pricing, customers, vehicles, orders, contracts, coverage, billing, claims and maintenance plans. Align definitions and authoritative sources across applications and enterprise data platforms.

• Engineer preventive data quality controls. Partner with application teams to implement validation for product eligibility, required customer and vehicle data, effective dates and downstream setup. Place controls at the earliest appropriate point in the process.

• Establish trusted and reconciled data products. Reverse-engineer current operational and financial reporting from source systems through legacy DB2/Hadoop transformations and downstream spreadsheets. Build automated reconciliation between legacy and target state data, explain material variances, and establish trusted datasets with Finance, Pricing and business data owners.

• Operationalize exception management. Create monitoring, alerts and actionable exception queues with clear reason codes, ownership, ageing and resolution status. Connect recurring failures to root-cause analysis and product backlog improvements.

• Support business data stewardship. Work with named business and domain owners, and stewards to maintain definitions, quality rules, reference data and issue-resolution processes. Make business decisions and exceptions explicit, documented and traceable.

• Deliver actionable operational and performance information. Develop reusable data products and reporting that help teams understand service sales performance, transaction completeness, quality trends, billing and claims outcomes, exceptions and service readiness, with drill-down to the underlying records and source lineage.

• Integrate data across current and target-state systems. Connect transactional applications, APIs, events and enterprise data platforms across CAPM, SAP, legacy DB2 and other sources. Apply appropriate approaches to matching, deduplication, normalization, reference-data management and historical continuity.

• Support data-platform modernization. Map current-state data lineage and business logic across DB2, Hadoop and business-managed reporting to target-state sources including CAPM, SAP and remaining legacy systems. Identify data gaps, undocumented transformations and business rules that must be preserved, deliberately changed or retired during migration.

• Own production reliability and traceability. Establish testing, observability, lineage, documentation, version control and CI/CD practices. Ensure data can be traced from business output back through transformations to authoritative source records, and support production incidents and the technical lifecycle of the data solutions you build.

• Lead technical delivery. Influence solution architecture and engineering standards, mentor other practitioners, and coordinate delivery across Product, Engineering, Data and business teams.

• Measure data and business improvement. Define and monitor outcomes such as Finance-approved reconciliation, data completeness and quality, successful contract-to billing/plan reconciliation, fewer invalid transactions, reduced manual intervention and faster exception resolution. Communicate findings and recommendations clearly to technical and business audiences.

Required Qualifications

• Recent hands-on experience designing, building and operating production data pipelines, data models and operational data products across enterprise systems.

• Advanced SQL and Python skills, supported by sound software-engineering practices.

• Strong data discovery and profiling skills, including identifying grain, keys, relationships, duplicates, missing data, temporal behavior, undocumented transformations and data quality patterns in unfamiliar datasets.

• Strong business analysis skills, including process mapping, requirements elicitation, business rule definition, root-cause analysis and acceptance criteria.

• Experience translating business concepts into logical and physical data models, including entities, relationships, identifiers, grain, temporal behavior and lifecycle states.

• Experience implementing data quality controls, reconciliation, monitoring and exception management processes for transactional, operational or financial/commercial data, including investigating and explaining discrepancies across systems and reporting layers.

• Experience integrating data through APIs, batch pipelines, change data capture or event driven patterns.

• Experience with modern cloud data platforms and distributed processing.

• Strong understanding of data lineage, testing, observability, access controls and production support.

• Ability to work with business owners and data stewards to agree definitions, resolve ambiguity and establish accountable data-quality processes.

• Experience leading technical work, defining solution approaches and influencing architecture across cross-functional teams.

• Ability to investigate issues across application, data and reporting boundaries, reverse engineer unfamiliar legacy logic, identify root causes and carry improvements through to measurable outcomes.

• Clear written and verbal communication skills, including the ability to explain technical findings and trade-offs to business stakeholders.

• Demonstrated ownership, independent judgement and experience mentoring other data practitioners. Desired Skills and Experience

• Experience with service products, configuration and pricing, order management, contracts, subscriptions, entitlements, maintenance or warranty processes.

• Experience working with Finance, Accounting or Pricing teams to reconcile commercial or financial data and establish authoritative business reporting.

• Experience with Databricks, Hadoop, Azure data services and modern lakehouse architectures, including curated data products and governed transformation patterns.

• Experience with Palantir Foundry, including data pipelines, ontology/semantic modelling and operational or analytical applications.

• Familiarity with transactional and financial platforms such as Pega, SAP and legacy mainframe/DB2 environments, including modernization or migration scenarios. • Experience with domain-driven design, master data management, semantic modelling and data governance.

• Experience implementing reconciliation across multiple systems and linking operational exceptions to support or workflow platforms such as ServiceNow.

• Experience in commercial vehicle, automotive, industrial B2B or dealer/distributor environments.

• Experience applying statistical methods or AI to anomaly detection, exception classification or operational decision support.

• A relevant advanced degree in computer science, engineering, information systems, analytics or a related discipline. Minimum Requirements

• Bachelor's degree and at least 2 years of data engineering, data quality and/or analytics or statistical analysis experience OR

• Master's degree and at least 1 year of data engineering, data quality and/or analytics or statistical analysis experience OR

• At least 5 years of data engineering, data quality and/or analytics or statistical analysis experience