Data Engineer

Job Description

Job Description

Description:

Job Role

We are seeking a skilled and experienced Data Engineer to join our team. The ideal candidate will play a critical role in designing, developing, and maintaining our Claims Data Warehouse (CDW), which is built on Snowflake and fed by Azure-based data services. The role requires deep, hands-on expertise in Snowflake, strong working knowledge of the Azure data ecosystem, and extensive experience with medical claims data, including professional, institutional, and pharmacy claims, eligibility, and provider data. You will combine SQL, data modeling, and performance optimization skills with a solid understanding of how healthcare claims are adjudicated, adjusted, and reported, to ensure the efficiency, scalability, and reliability of our data infrastructure. 

Responsibilities Include:

 

• Design, build, and optimize the Claims Data Warehouse in Snowflake, including databases, schemas, tables, views, stages, file formats, and workloads for loading, transforming, and sharing data. 
• Build continuous and batch ingestion into Snowflake using Snowpipe, COPY INTO, external stages, and Azure Blob Storage/ADLS Gen2 landing zones. 
• Implement incremental and near-real-time processing with Snowflake Streams, Tasks, and Dynamic Tables, including change data capture (CDC), schema drift handling, and error-recovery patterns to keep pipelines reliable. 
• Implement medallion architecture (Bronze, Silver, and Gold layers) in Snowflake, from raw claim file ingestion through standardized, deduplicated claim data to curated, analytics-ready marts. 
• Tune Snowflake performance and cost by right-sizing virtual warehouses, using Query Profile, clustering keys, search optimization, materialized views, and resource monitors, and by resolving latency, queuing, and storage issues. 
• Enforce Snowflake security and governance, including role-based access control (RBAC), dynamic data masking, row access policies, object tagging, and secure data sharing, to protect PHI and meet HIPAA requirements. 
• Use Snowflake Time Travel, Fail-safe, zero-copy cloning, and replication to support backup, recovery, testing, and business continuity/disaster recovery (BCDR) strategies. 
• Write efficient, well-documented SQL (and Snowpark or stored procedures where appropriate) to support extraction, transformation, and loading (ETL/ELT) of large claims data volumes. 
• Model claim header and claim line structures and manage the full claim lifecycle, including adjudication status, adjustments, reversals, voids, resubmissions, denials, and coordination of benefits (COB), so that the most current and accurate version of each claim is always available. 
• Apply and maintain healthcare code sets and reference data such as ICD-10-CM/PCS, CPT/HCPCS, revenue codes, DRG, NDC, place of service, type of bill, and modifiers, and map them accurately across source systems. 
• Develop claims-based data products to support utilization management, case management, cost and trend analysis, PMPM reporting, provider and network analytics, quality measures (e.g., HEDIS), risk adjustment, and financial reporting such as incurred-but-not-reported (IBNR) and claim run-out analysis. 
• Perform regular claims data quality and reconciliation checks, including financial balancing of billed, allowed, and paid amounts, duplicate claim detection, eligibility-to-claim matching, and completeness and timeliness validation against source systems. 
• Partner with claims, medical management, actuarial, finance, and analytics stakeholders to translate business and clinical requirements into well-modeled, trusted data. 
• Build, schedule, and monitor ETL/ELT pipelines using Azure Data Factory (ADF) to move claims and cases data from source systems, SFTP, and payer/vendor files into Snowflake, with parameterization, alerting, and retry logic. 
• Use Azure services such as ADLS Gen2/Blob Storage, Azure Key Vault, Microsoft Entra ID, Azure Monitor, and Azure SQL/SQL Server to build secure, observable, and cost-effective data platform components. 
• Maintain and migrate existing SQL Server Integration Services (SSIS) packages as needed, and help modernize them into ADF and Snowflake-based patterns. 
• Contribute to CI/CD, version control (Git, Azure DevOps), and infrastructure-as-code practices for Snowflake objects and Azure data pipelines. 
• Use AI tools such as Snowflake Cortex, GitHub Copilot, Microsoft Copilot in Fabric, and LLM-based assistants to speed up pipeline development, code generation, documentation, testing, and troubleshooting. 
• Optimize database performance by analyzing, monitoring, and fine-tuning SQL queries and database structures across Snowflake and SQL Server environments. 
• Develop and maintain data pipelines to ensure seamless data flow between various source systems and the EDW, and work with development teams to integrate data services with applications and business systems. 
• Monitor and maintain the health of the platform, addressing issues related to performance, latency, data freshness, and storage, and manage database security, backup, and recovery strategies to ensure data integrity and availability. 
• Keep up to date with emerging trends and best practices in Snowflake, cloud data engineering, healthcare data standards, and SQL optimization. 

Requirements:

 

• Bachelor's degree in computer science, Information Technology, or a related field. 
• Proven experience as a Data or Database Engineer with 10 or more years of experience in a healthcare environment, with a focus on SQL and Enterprise Data Warehousing. 
• Hands-on experience with Snowflake (Required), including Snowpipe, Streams and Tasks, Dynamic Tables, COPY INTO, RBAC, masking and row access policies, warehouse sizing and performance tuning, and secure data sharing. 
• Experience implementing medallion architecture (Bronze, Silver, and Gold layers) in Snowflake or another cloud data warehouse/lakehouse environment. 
• Extensive hands-on experience with medical claims data, including claim header/line structures, adjustments and reversals, and eligibility/enrollment data. 
• Working knowledge of X12 EDI healthcare transactions (837, 835, 834, 270/271, 276/277) and claim forms (CMS-1500, UB-04). 
• Strong command of healthcare code sets and reference data, including ICD-10, CPT/HCPCS, revenue codes, DRG, NDC, and place of service. 
• Experience building claims-based analytics and reporting, such as utilization, cost and trend, PMPM, risk adjustment, and quality measures (e.g., HEDIS) 
• Hands-on experience with Azure Data Factory (ADF) and Azure data services such as ADLS Gen2/Blob Storage, Key Vault, and Azure DevOps. 
• Experience with SQL Server Integration Services (SSIS) and SQL Server (preferred), particularly in maintaining or migrating legacy packages. 
• Strong knowledge of data modeling and database design principles, including dimensional modeling and handling slowly changing dimensions and claim versioning. 
• Understanding of data security, privacy, backup, and recovery procedures (i.e., BCDR), including HIPAA and handling of PHI/PII. 
• Experience with Git-based version control and CI/CD (Azure DevOps or similar). 
• Strong problem-solving skills and attention to detail. 
• Knowledge of Agile methodologies and experience working in Agile environments. 
• Strong communication skills working across business functions, including clinical, claims, and finance teams. 
• Manage multiple projects, issues, and deliverables in a fast-paced environment. Able to multi-task and prioritize work to meet deadlines. 
• Excellent listening, critical thinking, written and verbal skills. 
• Works well within a team environment. 
• Experience working in a professional environment with clinical managed care concepts, practices, and procedures preferred (e.g., utilization management, case management, prior authorization, provider networks, capitation). 

Benefits

Some of the benefits we offer our team are:

• Paid Time Off
• Paid Parental Leave
• Medical, dental, vision benefits
• 401K with company match
• Short- and Long-Term Disability
• Group Life Insurance
• Tuition Reimbursement
• Pet Insurance
• Professional development opportunities
• Business Casual work environment

ProgenyHealth positively recruits people from diverse backgrounds, including individuals with disabilities - if you need an accommodation to interview, please contact us by email: View email address on ziprecruiter.com
All qualified applicants will receive consideration for employment without regard to race, color, sex, sexual orientation, gender identity, religion, national origin, disability, veteran status, or other legally protected status.