Lead Data Engineer

***MUST HAVE CURRENT HEALTHCARE EXPERIENCE***

As a Lead Data Engineer, you will play a hands-on technical leadership role in evolving the data capabilities that power our client's solutions. You will lead complex initiatives across two closely connected areas: onboarding imperfect customer data for use within our products, and building scalable, reusable data processing capabilities that make data integrations increasingly automated, reliable, and repeatable. As a senior technical leader on the team, you will tackle complex and ambiguous problems, guide technical approaches and engineering practices, mentor other engineers, and bring a strong point of view to architectural decisions. Success requires technical depth, strong written and verbal communication, and curiosity about our customers’ data to recognize when recurring customer needs should evolve into reusable product capabilities.

Key Responsibilities

• Own the most complex and ambiguous engineering problems and manage multiple concurrent initiatives, driving solutions from technical design through implementation, testing, and delivery.

• Provide technical mentorship to the Data Engineering team, guiding architecture, engineering practices, technical decision-making, quality, and adoption of standards and best practices.

• Lead the onboarding of customer data, designing and evolving data models and ELT processes to handle imperfect source data while mapping to maximize our solutions.

• Design and build scalable and automated data processing capabilities that transform recurring data needs into reusable and reliable backend services, frameworks, and processing patterns.

• Bring curiosity to how customers create and use their data, identifying patterns across customers, and determining when solutions should evolve into reusable capabilities.

• Regularly evaluate alternative data architectures, models, and technologies, applying sound technical judgment to assess tradeoffs and recommend new approaches.

• Enforce best practices for design, documentation, and decision-making that drive shared understanding across teams. Articulate problems, solutions, tradeoffs, risks, recommendations.

• Design and build automated data profiling, validation, and quality monitoring services, including published metrics for stakeholders to understand the quality and shape of customer data sets.

• Drive continuous improvement to our data architecture, tooling, workflows, and engineering practices, including leveraging AI and other emerging technologies to increase automation, scalability, observability, performance, and maintainability.

• Create and manage data capability roadmaps. Propose recommended investments, priorities, dependencies, and expected outcomes to company leadership and stakeholders.

• Raise team effectiveness through mentorship, design, code reviews, knowledge sharing, feedback, collaboration, and transparent communication of priorities, status, and risks.

Required Skills & Qualifications

• Bachelor's degree in computer science or a related field, or equivalent experience, with 10+ years of relevant engineering experience.

• Demonstrated technical leadership , including mentoring, establishing engineering practices, guiding technical decisions, and raising overall team capability.

• Experience managing and developing a small engineering team (3-5 engineers) , including coaching, performance management, and supporting individual growth and development.

• Demonstrated ability to solve complex engineering problems, translate business requirements into technical solutions, evaluate tradeoffs, and lead initiatives from design through delivery.

• Experience designing, building, and evolving scalable and reusable data platforms including leading significant technology and architecture modernization efforts.

• Deep hands-on experience designing and building scalable ELT pipelines, data models , and transformations using SQL , Python , Linux , and relational, MPP, and file-based data architectures.

• Hands-on experience with modern ELT orchestration frameworks (e.g., Airflow, Astronomer, Dagster, dbt, Windmill), including operating, monitoring, and optimizing large scale data pipelines.

• Experience integrating complex and imperfect data from diverse sources, including understanding source-data assumptions, resolving quality and modeling challenges, and effective data mapping into standardized data models.

• Strong written and verbal communication skills , with demonstrated ability to articulate technical problems, solution alternatives, assumptions, tradeoffs, and recommendations.

• Experience working with healthcare data and demonstrated curiosity about users, their data, and the business problems that data is intended to solve.

Preferred Skills & Qualifications

• Experience developing and communicating technical or capability roadmaps based on recurring customer, product, or engineering needs.

• Experience modernizing legacy data architectures and migrating workloads from relational database-centric architectures to file-based, distributed, or cloud-native data architectures.

• Experience working in a SaaS product environment where data engineering supports both customer implementations and reusable product capabilities.