Senior Data Engineer (Banking Domain)

Experienced Senior Data Engineer to support large scale data platform modernization initiatives within a regulated banking environment.

The role focuses on designing and building enterprise-grade in-house frameworks, supporting high-volume batch and CDC-based incremental processing using Cloudera platform, and enabling ongoing Google Cloud Platform (GCP) modernization efforts.

Technology & Skill Requirements

• Apache Spark (PySpark and/or Scala) in large-scale production environments

• Cloudera Hadoop ecosystem (HDFS, Hive, YARN, Spark on Cloudera)

• Strong SQL expertise with complex transformations, performance tuning, and reconciliation logic

• Enterprise RDBMS experience with Oracle and MS SQL Server

• Batch ingestion, incremental ingestion, and CDC processing patterns

• CDC concepts and tooling (tool-agnostic: GoldenGate, Debezium, or equivalent)

• Data merge, deduplication, watermarking, checkpointing, and SCD handling

• Google Cloud Platform services including Dataproc , Composer and Dataplex

• Hybrid on prem to cloud data architecture and migration patterns

• Metadata-driven framework development and data quality validation techniques

• CI/CD pipeline implementation using enterprise tooling (GitHub Actions, Jenkins, DevOps)

• Git-based development workflows, code reviews, and automated testing practices

• Experience using Copilot or similar AI-assisted development tools safely and effectively in enterprise environments

• Logging, monitoring, alerting, and operational readiness practices

• Secure coding, access control, and compliance-aware development

• Documentation of design artifacts, runbooks, and operational procedures