Senior Data Engineer - GCP

Overview:

We are looking for a Senior Data Engineer with hands on experience in Google Cloud platform (GCP) resources such as BigQuery, Dataflow, PubSub, Cloud Composer, Cloud Run (native) and Cloud Storage (GCS). Knowledge and setting up of Data Transformation and integration using Data Build Tool (DBT) is also a requirement to support, scheduling, and analytics flows in the GCP environment.

The ability to co-ordinate with multiple offsite client teams to gather requirements in initial phase and also provide hands-on technical support during offshore hours is required.

• Liaison with client teams from multiple geographies to clarify requirements and get stakeholder buy-in.

• Setup appropriate Data Model for requirements and design data pipelines for implementation.

• Provide necessary documentation to support development teams.

• Design and implement data pipelines in GCP for ingesting and transforming data from SAP and AS400 systems to BigQuery using GCP native resources and DBT.

• Optimize ETLELT workflows for performance and scalability.

• Implement data quality checks, schema validation, and error handling.

• Support batch and near-real-time data flows for operational and analytics use cases.

• Strong experience with GCP (BigQuery, Dataflow, Cloud Composer, Cloud Run (native) and Cloud Storage (GCS).

• Proficiency in SQL and data analysis.

• Experience with PubSub.

• Familiarity with data governance and security best practices.

• Ability to work with large datasets and optimize for performance.

• Familiarity with CICD (Github and Terraform preferred). Knowledge of JSON, XML.

• Data Modelling concepts and Data Pipeline design