GCP Data Engineer

GCP Data Engineer

We are seeking a highly skilled GCP Data Engineer to design, develop, and optimize scalable data solutions on Google Cloud Platform (GCP). The ideal candidate will have strong expertise in building robust batch and streaming pipelines, implementing modern data architectures, and enabling reliable, high-quality data platforms for analytics, reporting, and machine learning use cases.

Key Responsibilities

Data Engineering & Pipeline Development

• Design, build, and optimize scalable batch and real-time (streaming) data pipelines using GCP native services.
• Develop and maintain data ingestion frameworks leveraging tools such as Pub/Sub, Dataflow, and Cloud Storage.
• Implement data transformation pipelines using BigQuery, dbt, and Python-based workflows.
• Ensure efficient handling of large-scale structured and unstructured datasets.
• Design and implement high-performance data models for cloud-based data lakes, data warehouses, and analytics platforms.
• Optimize data schemas and partitioning strategies in BigQuery for performance and cost efficiency.
• Support modern architectures such as medallion (bronze/silver/gold) layers and lakehouse patterns.

Development & Coding

• Write advanced SQL queries for transformation, validation, and analytics.
• Develop scalable data processing logic using Python and/or Apache Beam.
• Build reusable, modular, and maintainable code for data workflows.

Data Quality, Observability & Reliability

• Implement and maintain data quality checks, validation rules, and anomaly detection frameworks.
• Enable data observability through monitoring, logging, and alerting mechanisms.
• Ensure highly reliable data pipelines with fault tolerance and error handling strategies.

ETL/ELT Modernization

• Support migration and modernization efforts from legacy ETL tools (e.g., Talend) to GCP-native ELT frameworks (dbt).
• Optimize existing pipelines for performance, scalability, and maintainability in cloud environments.
• Drive adoption of ELT best practices using BigQuery as the compute engine.

Collaboration & Stakeholder Engagement

• Collaborate with data architects, business analysts, and machine learning teams to deliver trusted datasets.
• Translate business requirements into scalable data solutions.
• Provide technical guidance and support for downstream analytics and reporting use cases.

Best Practices & Governance

• Drive adoption of best practices in cloud data engineering, CI/CD, and DevOps.
• Implement secure data access controls using IAM roles, policies, and governance frameworks.
• Follow standards for code quality, version control (Git), and automated deployments.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• 4+ years of experience in data engineering or data platform development.
• Hands-on experience with Google Cloud Platform (GCP) services:
• BigQuery
• Dataflow
• Pub/Sub
• Cloud Storage
• Strong proficiency in SQL and Python.
• Experience with dbt (Data Build Tool) or similar ELT frameworks.
• Experience building batch and streaming data pipelines.

Preferred Skills

• Experience with Apache Beam or Spark.
• Familiarity with Talend or other ETL tools and migration to cloud-native solutions.
• Knowledge of data lakehouse architectures and modern data stack.
• Experience with CI/CD tools (e.g., GitHub Actions, Cloud Build, Jenkins).
• Understanding of data security, governance, and compliance standards.
• Exposure to machine learning data pipelines and feature engineering.

Key Competencies

• Strong problem-solving and analytical skills
• Ability to work in cross-functional teams
• Excellent communication and documentation skills
• Focus on performance optimization and scalability
• Attention to data quality and reliability