Consultant, Data Engineer

About the role

What you will do

• Build and maintain batch and incremental ETL pipelines in Databricks using Python, PySpark and Spark SQL.
• Ingest data from relational databases, APIs, flat files and cloud storage (Amazon S3, Azure Data Lake Storage) into Delta Lake tables.
• Implement bronze, silver and gold layers following medallion architecture patterns.
• Write and tune SQL for transformations, data validation and reporting tables.
• Add data quality checks, handle schema changes and troubleshoot failed jobs.
• Schedule and monitor workloads with Databricks Workflows, working within Unity Catalog permissions.
• Tune Spark jobs for performance and cost, including partitioning, file sizing and cluster configuration.
• Use Git for version control, take part in code reviews and contribute to CI/CD pipelines.
• Document pipelines, source-to-target mappings and runbooks so client teams can run what you build.
• Join client meetings to clarify requirements, report status and walk through your work.

What you bring

• 1 to 3 years of professional experience in data engineering, software engineering or a related technical role.
• Bachelor's degree in computer science, information systems, engineering, mathematics or a related field, or equivalent experience.
• Hands-on experience with Databricks on a professional or substantial academic project.
• Working proficiency in Python for data processing.
• Experience building transformations with Apache Spark and PySpark (DataFrames and Spark SQL).
• Strong SQL, including joins, window functions, CTEs and aggregations.
• Understanding of ETL concepts such as incremental loads, slowly changing dimensions, dimensional modeling and data quality checks.
• Familiarity with at least one cloud platform (AWS, Azure or Google Cloud).
• Clear written and verbal communication, and comfort explaining technical work to non-technical stakeholders.
• Authorization to work in the United States [confirm sponsorship policy].

Nice to have

• Databricks Certified Data Engineer Associate certification.
• Experience with Delta Lake features such as MERGE and time travel, or with Lakeflow Declarative Pipelines (formerly Delta Live Tables).
• Exposure to Unity Catalog, dbt, Apache Airflow or Azure Data Factory.
• Experience with Terraform or CI/CD tools such as GitHub Actions or Azure DevOps.
• Prior consulting or client-facing experience.
• Background in financial services, healthcare or media data.