Principal Data Platform Engineer

Role Description

We are currently looking for a Principal Data Platform Engineer (Databricks) to join a modern data platform initiative for healthcare and life sciences clients. This role will focus on owning end-to-end architecture and delivery of scalable data solutions, working closely with data architects, analysts, and client stakeholders.

Location: Remote, United States

Key Responsibilities:

• Own end-to-end architecture and delivery of scalable data solutions, with strong emphasis on Databricks-based (or comparable Snowflake / Microsoft Fabric) platforms and modern cloud ecosystems.
• Lead the design and implementation of data pipelines, data models, and transformation frameworks supporting analytics, reporting, and advanced use cases.
• Serve as the primary client-facing technical lead, building trusted relationships and guiding stakeholders through complex data decisions.
• Translate ambiguous business requirements into clear technical architectures and delivery plans.
• Establish and enforce best practices across data engineering — ingestion, pipeline orchestration, testing, optimization — and DevOps/CI-CD.
• Drive platform strategy and architecture decisions, including lakehouse design, medallion architecture, and governance frameworks.
• Lead and mentor delivery teams, providing technical guidance, code reviews, and hands-on support.
• Collaborate with cross-functional teams — data architects, analysts, client stakeholders — to ensure alignment and value delivery.
• Identify risks and proactively address challenges to ensure high-quality, on-time delivery.
• Contribute to internal capability building: reusable frameworks, accelerators, and thought leadership.
• Support business development by shaping technical solutions and contributing to proposals and client discussions.

Qualifications

• 7+ years of data engineering experience, with clear progression into technical leadership and architecture ownership.
• Deep expertise in Databricks and modern lakehouse architectures, including Delta Lake and Spark-based processing (comparable Snowflake or Microsoft Fabric experience also considered).
• Advanced SQL and Python skills, with strong experience building and optimizing large-scale data pipelines.
• Hands-on experience with cloud platforms (AWS, Azure, or GCP), including data services and infrastructure design.
• Solid understanding of data modeling concepts, ETL/ELT patterns, and distributed data processing.
• Experience with data ingestion pipelines, orchestration tools (e.g., Airflow), and transformation frameworks (e.g., dbt).
• Hands-on experience with CI/CD and DevOps practices in a data engineering context.
• Proven ability to lead technical delivery while staying hands-on.
• Strong client-facing experience: requirements gathering, solution design, executive communication.
• Ability to navigate ambiguity, prioritize effectively, and drive clarity in complex environments.

Requirements

• Healthcare data experience (e.g., Epic, HL7, FHIR, claims data).
• Experience with infrastructure-as-code tools (e.g., Terraform).