Data Engineer

Overview We are looking for an experienced Data Engineer to join the Supply/Yield Data Engineering team and support a large-scale data modernization and analytics transformation programme through 2026. This role will focus on designing, developing, and maintaining data pipelines that bring data into Databricks, support Inventory and Yield analytics initiatives, and enable business-critical reporting within Tableau Cloud. The successful candidate will also contribute to the migration of reporting assets from Looker to Tableau while helping modernize existing legacy data engineering solutions. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines to ingest and transform data into Databricks. Build and support scalable data pipelines using AWS, Databricks, and Snowflake. Work closely with business and technical stakeholders to understand requirements and translate them into robust data solutions. Develop core data assets and datasets that support analytics and reporting consumed through Tableau Cloud. Support the Inventory ART programme by delivering Data Engineering solutions and increasing team capacity during migration activities. Enhance and optimize existing legacy data engineering jobs and processes. Contribute to the migration of reporting solutions from Looker to Tableau. Create and maintain technical documentation, specifications, and implementation guides. Perform testing, validation, and quality assurance activities to ensure data accuracy and reliability. Provide post-delivery support, incident resolution, and troubleshooting for production workloads. Implement Infrastructure as Code (IaC) solutions using Terraform. Build and manage CI/CD pipelines using Concourse or GitHub Actions to enable reliable and automated deployments. Help identify and mitigate risks, including reducing Single Points of Failure (SPOF) through documentation, automation, and knowledge sharing. Collaborate with fellow engineers to establish and promote engineering best practices. Required Skills & Experience Strong experience in Data Engineering and ETL/ELT development. Hands-on experience with Databricks. Strong SQL skills with experience working on large datasets. Experience building data pipelines within AWS environments. Experience with Snowflake data warehousing. Experience with Spark/PySpark. Hands-on experience with Terraform and Infrastructure as Code (IaC). Knowledge of CI/CD practices and tools such as GitHub Actions, Concourse, Jenkins, or similar. Experience supporting and troubleshooting production data pipelines. Strong understanding of testing, data quality, and operational support processes. Excellent problem-solving and collaboration skills. Nice to Have Experience with Tableau or Tableau Cloud. Experience supporting Looker to Tableau migrations. Experience working within Supply Chain, Inventory, Yield, or Manufacturing domains. Data modelling and analytics engineering experience. Experience modernizing legacy data platforms and applications. Technology Stack Databricks AWS Snowflake SQL Spark / PySpark Terraform GitHub Actions / Concourse Tableau Cloud Looker (Migration Support) Project Highlights Work on a strategic data modernization programme running through 2026. Build the core data pipelines powering Inventory, Supply, and Yield analytics. Support a major Looker-to-Tableau reporting migration. Develop cloud-native solutions using modern data engineering technologies. Play a key role in improving automation, reliability, scalability, and operational excellence across the data platform. #J-18808-Ljbffr