Data engineering Lead

We are seeking a Technical Lead to provide hands-on technical leadership for clinical data engineering, data platform development, and analytics delivery. This role is highly hands-on and will work closely with and coordinate the India-based engineering team, leading the design and execution of scalable data pipelines, lakehouse architecture, cloud services integration, and day-to-day delivery across active clinical data initiatives.

The ideal candidate will have strong experience with Databricks Lakehouse, Delta Lake, Unity Catalog, Medallion architecture, SQL Warehouses, AWS services, data engineering delivery, agile execution, and stakeholder management in a life sciences or regulated data environment.

Key Responsibilities

Serve as the technical lead for clinical data engineering and platform delivery.

Own architecture and implementation across Databricks Lakehouse, Delta Lake, Unity Catalog, Medallion architecture, and SQL Warehouses.

Lead end-to-end delivery of data pipelines, curated datasets, data products, and analytics-ready data assets.

Work hands-on alongside and coordinate the India-based data engineering team, providing day-to-day technical direction and coordination across time zones.

Provide technical direction on pipeline design, data modeling, performance optimization, security, scalability, and production readiness.

Work with AWS services such as S3, EMR/Athena, Secrets Manager, IAM, and related cloud platform components.

Drive sprint planning, backlog management, technical design reviews, code reviews, release planning, and delivery governance.

Partner with Clinical Operations, Biostatistics, Data Management, IT, data managers, and business stakeholders to translate requirements into technical solutions.

Coordinate cross-functional delivery across data engineering, analytics, AI/ML, platform, QA, and business teams.

Ensure engineering outputs meet quality, security, compliance, performance, and maintainability expectations.

Support issue resolution across data ingestion, transformation, pipeline failures, data quality defects, and platform dependencies.

Provide technical oversight for automation, AI/ML-enabled workflows, and human-in-the-loop data operations where applicable.

Required Qualifications

10–12 years of experience in data engineering, data architecture, analytics engineering, or cloud data platform delivery.

Strong hands-on experience with Databricks Lakehouse, including:

Delta Lake

Unity Catalog

Medallion architecture

SQL Warehouses

Notebook/job orchestration

Data pipeline performance tuning

Working knowledge of AWS services, including S3, EMR/Athena, Secrets Manager, IAM, and cloud-native security patterns.

Proven experience working hands-on while coordinating distributed engineering teams, including offshore/India-based engineers, and owning end-to-end technical delivery while remaining hands-on.

Strong understanding of data pipeline design, batch processing, data transformation, data modeling, and analytics enablement.

Experience with agile delivery, sprint ceremonies, backlog management, release management, and technical execution governance.

Ability to work in cross-functional delivery models involving technical teams, business users, data managers, and client stakeholders.

Strong stakeholder management and client-facing communication skills.

Ability to translate business and clinical data requirements into scalable technical architecture and implementation plans.

Preferred Qualifications

Experience in pharma, biotech, CRO, clinical operations, or life sciences data environments.

Familiarity with clinical data domains, study data, lab data, safety data, EDC data, or biostatistics workflows.

Experience working with AI/ML, automation, or metadata-driven data pipelines.

Knowledge of data governance, data quality, lineage, auditability, and regulated data controls.

Experience with CI/CD, Git-based development, DevOps practices, and production support models.

Experience managing distributed onsite/offshore delivery teams.