Data Engineer (Azure Databricks)

Summary

Seeking a Data Engineer with 7+ years of experience to build and support cloud-based data platforms using Azure and Databricks . The role focuses on developing scalable data pipelines, supporting analytics and reporting, and enabling data-driven decision-making in a healthcare environment. Key Responsibilities

• Build and maintain ETL/ELT data pipelines.
• Develop data solutions using Azure Databricks, ADF, and ADLS.
• Create and optimize PySpark and SQL-based data processing frameworks.
• Implement Delta Lake and Lakehouse architectures.
• Support data migration, modernization, and cloud transformation projects.
• Ensure data quality, reliability, and performance.
• Build data models and curated datasets for reporting and analytics.
• Implement CI/CD, monitoring, and DevOps best practices.
• Collaborate with business users, architects, and product teams in an Agile environment.

Required Skills

• Azure Databricks
• Azure Data Factory (ADF)
• Azure Data Lake Storage (ADLS Gen2)
• PySpark
• Python
• SQL
• Delta Lake
• Data Modeling
• ETL/ELT Development
• Azure Cloud Platform
• Git & CI/CD
• Agile/Scrum

Preferred Skills

• Azure Synapse Analytics
• Power BI
• Azure DevOps
• Infrastructure as Code (IaC)
• FHIR/HL7
• AI/ML Data Pipelines

Required Experience

• 7+ years in Data Engineering
• 5+ years with Azure Data Technologies
• 2+ years in Healthcare, Payer, Health Plan, or Managed Care domains
• Experience with large-scale data pipelines and integration solutions

Nice to Have

• Claims Data
• Membership Data
• Provider Data
• Care Management Data