Data Engineer

THE POSITION

You’ll be a key member of VaxCare’s Product Group, joining our Data Engineering team and reporting to our Data Engineering Lead. We are seeking a motivated and capable Data Engineer to join our team. As a Data Engineer, you will contribute to the design, development, and management of our data processing and analytics infrastructure. The ideal candidate will have hands‑on experience working with Spark and Databricks, a solid foundation in data engineering principles, and a desire to grow into a senior technical contributor. Responsibilities Develop and maintain Delta Lake‑based data pipelines using Databricks Workflows, Delta Live Tables (DLT), and Unity Catalog for enterprise data governance Build ELT/ETL pipelines using medallion architecture (bronze/silver/gold layers) supporting both batch and streaming workloads with Auto Loader and Structured Streaming Implement lakehouse solutions leveraging Delta Lake ACID transactions, Z‑ordering, liquid clustering, and partitioning strategies Support CI/CD pipelines for data workflows using Git integration and Databricks Asset Bundles Contribute to data quality frameworks using Delta Live Tables expectations and custom PySpark validation with automated alerting and SLA monitoring Create materialized views and incremental refresh strategies for optimized query performance Collaborate with data scientists, ML engineers and analysts to support feature engineering pipelines and MLOps workflows Participate in code reviews and contribute to technical design discussions Implement data observability, monitoring using Databricks SQL, Lakeview dashboards, and alerting frameworks Support cost optimization efforts leveraging Photon engine, serverless compute, and platform best practices Troubleshoot and resolve issues related to distributed computing, data skew, and performance bottlenecks Contribute to technical documentation including data contracts, runbooks, and data catalog metadata in Unity Catalog Follow DataOps best practices including testing strategies, performance tuning, and data platform engineering principles Stay current with lakehouse architecture trends and emerging technologies to continuously improve our data infrastructure Experience and Qualities Desired Bachelor’s degree in Computer Science, Data Engineering, Engineering, or related technical field OR equivalent practical experience Master’s degree or relevant industry certifications (Databricks Certified Data Engineer Associate, Azure Data certifications) are a plus Experience Must be located in the Greater Orlando / Boston area. 3‑5 years of data engineering experience with 1+ years hands‑on production experience building data pipelines on Databricks and Apache Spark Experience contributing to lakehouse architecture implementations Technical Skills Programming & Languages: Strong proficiency in Python (PySpark, pandas) and SQL (complex queries, window functions, CTEs, query optimization) Experience with Spark SQL, Delta Lake SQL, and Databricks SQL Apache Spark Expertise Working knowledge of Apache Spark including: Performance fundamentals (partitioning, broadcast joins, data skew handling, caching strategies) Delta Lake features (ACID transactions, time travel, MERGE operations, CDC, liquid clustering) Databricks Platform Hands‑on experience with Databricks including: Delta Live Tables (DLT) for declarative pipeline development Unity Catalog for data governance, access control, and lineage tracking Databricks Workflows and orchestration Basic understanding of cluster configuration and cost‑aware compute selection Databricks SQL and Lakeview dashboards Data Architecture & Modeling Solid understanding of data modeling techniques: Dimensional modeling (star schema, fact/dimension tables) Medallion architecture (bronze/silver/gold layers) Slowly Changing Dimensions (SCD) implementations Strong SQL skills including query optimization and performance tuning Familiarity with modern lakehouse patterns and understanding of lakehouse vs. traditional data warehouse trade‑offs DevOps & DataOps Familiarity with DevOps/DataOps practices: Git workflows (branching strategies, pull requests, code reviews) CI/CD pipelines for data workflows (GitHub Actions, Azure DevOps, Jenkins) Testing strategies (unit tests, integration tests, data quality tests) Basic monitoring and observability (logging, alerting) Collaboration & Growth Works independently to deliver high‑quality, well‑tested solutions with meaningful impact on the team’s data infrastructure Takes ownership of assigned projects and drives them to completion with minimal oversight Strong communication and collaboration skills in cross‑functional team environments Proactive in identifying problems and proposing solutions, even outside immediate area of responsibility Demonstrates initiative in expanding technical depth and breadth, with a trajectory toward senior‑level engineering Open to feedback and committed to continuous improvement #J-18808-Ljbffr VaxCare