Data Platform Engineer

Data Platform Engineer

Department: Data & Analytics - Location: Hybrid - Employment Type: Full-Time

Position Overview

We are seeking an experienced Data Engineer to join our growing team and play a central role in shaping our data strategy. This individual will be responsible for building, optimizing, and governing our Databricks-based data platform hosted on AWS connecting disparate data sources, enabling reliable data pipelines, and ensuring that high-quality data is available to support business intelligence, analytics, and machine learning initiatives.

Key Responsibilities

Data Strategy & Platform Governance

• Partner with leadership to define and execute a comprehensive data strategy aligned with business goals.
• Establish standards and best practices for data ingestion, transformation, storage, and access across the organization.
• Drive adoption of data governance frameworks including data cataloging, lineage tracking, and quality monitoring.
• Champion a culture of data-driven decision-making by enabling self-service analytics capabilities.

Databricks Platform Administration

• Architect, configure, and maintain the Databricks environment hosted on AWS (EMR, S3, IAM, VPC, etc.).
• Manage Databricks workspaces, clusters, Unity Catalog, and access controls.
• Optimize compute and storage costs through cluster sizing, auto-scaling, and lifecycle management.
• Implement and maintain CI/CD pipelines for data workflows using tools

Data Integration & Pipeline Development

• Design and build scalable ETL/ELT pipelines to ingest data from a variety of sources including SaaS applications, databases, APIs, and streaming platforms.
• Leverage Delta Lake, Delta Live Tables, and Databricks Workflows to build reliable, incremental data processing solutions.
• Integrate cloud-native AWS services (Glue, Kinesis, EventBridge, Lambda) with the Databricks ecosystem.
• Ensure data pipeline reliability through monitoring, alerting, and automated recovery mechanisms.

Collaboration & Data Enablement

• Partner with data analysts, data scientists, and business stakeholders to understand data requirements and deliver fit-for-purpose datasets.
• Develop and maintain data documentation, including data dictionaries, runbooks, and architecture diagrams.
• Mentor junior team members on best practices in data engineering and cloud-native development.

Required Qualifications

Technical Skills

5+ years of hands-on data engineering experience in a cloud environment.

Strong proficiency with Databricks (Notebooks, Workflows, Delta Lake, Unity Catalog, SQL Warehouses).

Deep expertise with AWS services: S3, IAM, VPC, Glue, Redshift, RDS, Lambda, Kinesis, or EventBridge.

Proficiency in Python for data engineering workloads; SQL expertise required.

Experience designing and building data lake / lakehouse architectures.

Familiarity with data transformation tooling such as dbt, Apache Spark, or similar frameworks.

Knowledge of data modeling concepts (dimensional modeling, data vault, or medallion architecture).

Experience & Education

Bachelor's degree in Computer Science, Information Systems, Engineering, or equivalent practical experience.

Proven track record of delivering end-to-end data platform solutions at scale.

Experience with infrastructure-as-code tools such as Terraform is a strong plus.

Databricks Certified Associate or Professional certification preferred.

AWS Certified Data Analytics or Solutions Architect certification preferred

We are not able to consider candidates who will require sponsorship at any point during employment.

EEO/Vets/Women/Minorities/Disabled