Lead Data Platform Engineer

Overview

The Lead Data Platform Engineer is the highest individual contributor engineering role on the EDAP (Enterprise Data Analytics Platform) team. You will serve as the primary technical authority for the platform, owning the end-to-end architecture and driving the largest, most complex engineering initiatives across the organization. This is not a management role — you will not carry a headcount budget or run performance reviews — but your technical leadership will shape how an entire team of engineers do their work.

You will set the architectural vision for the platform, define engineering standards, and hold functional oversight over technical decisions made across the team. You will lead multi-quarter, cross-functional programs from inception to production, partner directly with engineering leadership and senior stakeholders, and represent the platform function in organizational-level planning. Engineers at every level will look to you for guidance, direction, and mentorship.

This role requires technical depth and leadership qualities. You are the person the team turns to when the problem is hard, ambiguous, or high stakes.

Key Responsibilities

Architecture & Technical Vision

• Own the technical architecture of the data platform end-to-end: ingestion, storage, transformation, orchestration, serving, and observability layers.

• Author and maintain the platform architectural vision document; lead quarterly architecture reviews to assess alignment with organizational goals and technology trends.

• Define and evolve the target-state architecture for the platform, establishing a multi-year technology roadmap in partnership with engineering leadership.

• Evaluate emerging technologies, frameworks, and make evidence-based adoption recommendations.

• Serve as the final technical escalation point for complex design questions, cross-team conflicts, and build-vs-buy decisions affecting the platform.

Large-Scale Project Leadership

• Lead the planning, execution, and delivery of multi-quarter platform initiatives involving multiple engineers, cross-functional dependencies, and significant organizational impact.

• Break down large, ambiguous programs into scoped workstreams; assign technical leads per workstream, define milestones, and manage cross-team dependencies and risk.

• Drive initiative kick-offs with clear problem framing, success criteria, architectural constraints, and delivery phasing — from 0-to-1 exploration through production hardening.

• Maintain stakeholder alignment throughout delivery: proactively communicate status, surface trade-offs, and escalate blockers to leadership before they become risks.

• Lead post-mortems and retrospectives for large initiatives; document and socialize lessons learned to raise the organizational bar on delivery excellence.

Functional Oversight & Engineering Standards

• Exercise functional oversight across the EDAP team: review technical designs, set quality gates, approve architectural decisions, and ensure consistency of implementation patterns.

• Define, document, and socialize platform engineering standards including coding conventions, testing requirements, CI/CD practices, schema design guidelines, and SLA frameworks.

• Establish and own the platform's technical review process (design review): triage incoming projects, chair review sessions, and ensure decisions are documented and traceable.

• Identify and drive resolution of technical debt, redundancy, and architectural drift that impede platform reliability, developer productivity, or scalability.

• Partner with security, compliance, and infrastructure teams to ensure platform systems meet governance, data privacy, and regulatory requirements by design.

Mentorship & Organizational Influence

• Serve as a primary technical mentor for junior and senior data platform engineers across the platform; provide structured coaching on system design, technical communication, and engineering judgment.

• Conduct and lead design reviews, architecture critiques, and technical deep-dives that strengthen the overall capability of the engineering team.

• Define what technical excellence looks like at each level of engineers and participate in calibration discussions.

• Represent the data platform in engineering-wide forums, all-hands, and external venues (conferences, open-source communities, recruiting events).

• Build a culture of documentation, reliability, and platform-as-a-product thinking across all data platform engineering functions.

• Participate in goal planning cycles as a technical voice; define engineering-led goals that improve platform reliability, developer experience, and data quality.

Required Qualifications

• 8–14 years of professional software or data engineering experience, with at least 3 years in a technical lead, principal, capacity on a data platform or distributed systems team.

• Demonstrated track record of leading large-scale, cross-functional data infrastructure projects from conception through production with measurable business impact.

• Deep expertise in distributed data systems: data warehouse/lakehouse architecture, streaming platforms, large-scale batch processing, and cloud-native data infrastructure.

• Expert-level proficiency in Python and SQL; strong working knowledge of at least one JVM language (Scala or Java) for Spark or Flink development.

• Deep knowledge of modern data warehouse and lakehouse platforms (Snowflake, BigQuery, Redshift, Databricks, or equivalent) including storage optimization, compute management, and cost governance.

• Strong command of open table formats (Apache Iceberg, Delta Lake, or Apache Hudi) and their trade-offs in production lakehouse architectures.

• Experience architecting and operating large-scale streaming pipelines with Apache Kafka, Kinesis, or Pub/Sub, including schema management, consumer group design, and exactly-once semantics..

• Exceptional communication skills: ability to write clearly, present confidently, and adapt technical depth to a wide range of audiences from individual contributors to high level leaders.

• Experience establishing engineering standards, review processes, and quality gates across multi-team engineering organizations.

Preferred Qualifications

• Experience designing and implementing federated data platform architectures at scale- like data mesh.

• Background in ML infrastructure, feature engineering platforms, or MLOps pipelines that intersect with the data platform layer.

• Familiarity with data governance, data cataloging, and lineage tooling (DataHub, OpenMetadata, Collibra, or Alation).

• Prior experience working at a company with a mature, scaled data platform serving hundreds of internal consumers or petabyte-scale data assets.

• Exposure to formal program management methodologies or experience driving large initiatives using structured delivery frameworks.

At Atlantic Health, our promise to our communities is; Anyone who enters one of our facilities will receive the highest quality care delivered at the right time, at the right place, and at the right cost. This commitment is also echoed in the respect, development and opportunities we give to our more than 22,000 team members. Headquarters in Morristown, New Jersey, we are one of the leading non-profit health care systems in the nation. Our facilities and sites of care include:

• Atlantic Health Morristown Medical Center, Morristown, NJ

• Atlantic Health Overlook Medical Center, Summit, NJ

• Atlantic Health Newton Medical Center, Newton, NJ

• Atlantic Health Chilton Medical Center, Pompton Plains, NJ

• Atlantic Health Hackettstown Medical Center, Hackettstown, NJ

• Atlantic Health Goryeb Children's Hospital, Morristown, NJ

• Atlantic Health CentraState Healthcare System, Freehold, NJ

• Atlantic Medical Group

• Atlantic Visiting Nurse

• Atlantic Mobile Health

• Atlantic Rehabilitation

We have more than 900 community-based healthcare providers affiliated through Atlantic Medical Group.

We have received awards and recognition for the services we have provided to our patients, team members and communities. Below are just a few of our accolades:

• Chosen for 17 years by Fortune as one of the magazine’s “100 Best Companies to Work For."

• Atlantic Health Morristown and Atlantic Health Overlook Named by Newsweek as two of the “World’s Best Hospitals” in 2026.

• Atlantic Health Morristown and Atlantic Health Overlook ranked within the top three hospitals in New Jersey by U.S. News & World Report’s 2025-2026 Best Hospital rankings.

• Atlantic Health scored four “A” grades by The Leapfrog Group in its Fall 2025 Hospital Safety Grades, performance measures reflecting errors, accidents, injuries and injections, as well as systems hospitals have in place to prevent harm.

• Atlantic Health Morristown and Atlantic Health Overlook are New Jersey's only hospitals to be named among America's 50 Best hospitals by Healthgrades in 2026.

• Named by Becker's Healthcare as one of the "165 Top Places to Work in Healthcare – 2026.

• Atlantic Health Morristown, Atlantic Health Overlook, Atlantic Health Chilton and Atlantic Health Newton all Forbes Top Hospitals for 2026.

• Named by Newsweek as one of America’s Greatest Workplaces for Inclusion & Diversity 2025.

• Atlantic Health rated LEVEL 9 - 2025 CHIME Digital Health Most Wired.

Atlantic Health offers a competitive and comprehensive Total Rewards package that supports the health, financial security, and well-being of all team members. Offerings vary based on role level (Team Member, Director, Executive). Below is a general summary, with role-specific enhancements highlighted:

Team Member Benefits

• Medical, Dental, Vision, Prescription Coverage (22.5 hours per week or above for full-time and part-time team members)

• Life & AD&D Insurance.

• Short-Term and Long-Term Disability (with options to supplement)

• 403(b) Retirement Plan: Employer match, additional non-elective contribution

• PTO & Paid Sick Leave

• Tuition Assistance, Advancement & Academic Advising

• Parental, Adoption, Surrogacy Leave

• Backup and On-Site Childcare

• Well-Being Rewards

• Employee Assistance Program (EAP)

• Fertility Benefits, Healthy Pregnancy Program

• Flexible Spending & Commuter Accounts

• Pet, Home & Auto, Identity Theft and Legal Insurance

Note: In Compliance with the NJ Pay Transparency Act (effective Sunday, June 1, 2025), all job postings will include the hourly wage or salary (or a range), as well as this summary of benefits. Final compensation and benefit eligibility may vary by role and employment status and will be confirmed at the time of offer.

EEO STATEMENT

Atlantic Health, Inc. is an equal employment opportunity employer and federal contractor or subcontractor and therefore abides by applicable laws to protect applicants and employees from discrimination in hiring, promotion, discharge, pay, fringe benefits, job training, classification, referral, and other aspects of employment, on the basis of race, color, religion, sex (including pregnancy, gender identity and sexual orientation), national origin, citizenship status, disability, age, genetics, or veteran