Data Engineer

Senior Data Engineer – Databricks, Snowflake

Position Summary: We are seeking a highly skilled and proactive Senior Data Engineer of experience in designing and building scalable data solutions across cloud platforms. This role demands deep expertise in Databricks, data pipeline architecture, and hands-on leadership in engineering teams. The ideal candidate will be instrumental in driving the development of robust data pipelines, mentoring junior engineers, and coordinating cross-functional efforts to deliver high-quality data products in the healthcare domain.

Key Responsibilities:

• Lead the design, development, and optimization of data pipelines using Databricks (PySpark, SQL) across Azure and GCP environments.
• Architect and manage Snowflake and BigQuery data warehouse solutions for performance, scalability, and cost-efficiency.
• Collaborate closely with stakeholders to understand data requirements and translate them into technical solutions.
• Perform advanced data analysis on healthcare datasets (claims, EMR, HL7, FHIR) to support strategic decision-making.
• Drive hands-on coordination of engineering tasks, ensuring timely delivery and quality standards.
• Mentor and guide junior data engineers, fostering best practices in coding, testing, and documentation.
• Implement and maintain data governance, lineage, and security using tools like Unity Catalog, Terraform, and GitLab CI/CD.
• Ensure compliance with healthcare data regulations (HIPAA, HITRUST) and internal security policies.
• Contribute to the evolution of data engineering standards, reusable frameworks, and automation strategies.

Required Skills & Experience:

• Experience in data engineering, with a strong focus on cloud-native data pipelines.
• Expert-level proficiency in Databricks, PySpark, SQL, and data orchestration.
• Proven experience with Snowflake, BigQuery, and healthcare data formats (FHIR, HL7, claims, EMR).
• Strong understanding of Azure and GCP cloud services including Data Factory, Synapse, Cloud Composer, and Google Cloud Storage.
• Hands-on experience with CI/CD, DevOps for data, and infrastructure-as-code.
• Excellent communication and leadership skills, with a track record of mentoring and team coordination.

Preferred Qualifications:

• Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.
• Certifications in Databricks, Snowflake, or GCP.
• Experience with orchestration tools like Airflow, dbt, or Cloud Composer.