Senior Specialist - Data Engineering

Role description

Job Description Developer Senior Developer Data Engineer Databricks on GCP

Service Type Development Services Build

Industry MultiNational FMCG

Environment Hyperscalerheavy GCPfirst Data Platform

Primary Skills Must Have

Databricks on GCP Core

Strong handson experience with Databricks GCP

Workspace clusters jobs and notebooks

Delta Lake ACID transactions time travel schema evolution

Medallion architecture Bronze Silver Gold layers

Experience with Databricks SQL SQL Warehouses for analytics workloads

PySpark Spark Core

Advanced development using PySpark Spark SQL

Performance tuning

Partitioning caching broadcast joins

Handling largescale distributed data processing

Debugging and optimizing Spark jobs in production

GCP Data Ecosystem Core

Strong integration with

Google Cloud Storage GCS for data lake storage

BigQuery for downstream analytics

Dataproc Dataflow nice to have for hybrid pipelines

Understanding of GCP networking IAM and service integration

Data Engineering ETL Core

Designing and building endtoend ETLELT pipelines

Handling batch and near realtime data processing

Data ingestion from enterprise systems SAP APIs files databases

Key Responsibilities

Data Pipeline Development

Design and develop scalable data pipelines using Databricks on GCP

Ingest transform and curate large datasets into analyticsready formats

Implement business logic using PySpark aligned with FMCG data domains

Lakehouse Engineering

Build and maintain Delta Lakebased architectures

Ensure data quality consistency and performance across layers

Enable downstream consumption via BigQuery and analytics tools

Performance Cost Optimization

Optimize Spark jobs for performance and cost efficiency

Reduce data scan volumes and improve query execution times

Collaborate with FinOps teams for costaware data processing

DataOps CICD

Implement CICD pipelines for Databricks using

GitHub GitLab Bitbucket

Deployment automation for notebooks and jobs

Version control and promote code across environments DevUATProd

Security Governance

Implement data governance using

Databricks access controls Unity Catalog if enabled

GCP IAM roles and permissions

Ensure compliance with enterprise data policies and regulations

Collaboration Delivery

Work closely with Data Analysts BI teams and business stakeholders

Translate business requirements into scalable data solutions

Participate in agile ceremonies and global delivery model