GCP Data Engineer

Senior Data Engineer (Spark Streaming & GCP) Job Summary We are seeking a highly skilled Senior Data Engineer with strong expertise in Apache Spark, Streaming Technologies, and Google Cloud Platform (GCP) to design, build, and optimize scalable data pipelines supporting analytics, reporting, and machine learning workloads. The ideal candidate will have extensive experience developing both batch and real-time data processing solutions using Spark, Kafka, and cloud-native data services. Required Experience 10+ years of overall IT experience 5+ years of recent hands-on GCP experience Strong experience building enterprise-scale data platforms and streaming architectures Required Skills Strong programming skills in Python and SQL Hands-on expertise with Apache Spark (PySpark, Spark SQL, DataFrames, Spark Streaming) Experience with Kafka, Pub/Sub, Flink, or other streaming technologies Strong knowledge of BigQuery, GCS, Data Lakes, and Data Warehousing concepts Experience designing and developing ETL/ELT pipelines Data modeling and performance optimization experience Experience with Airflow for workflow orchestration Knowledge of Snowflake, Redshift, or other cloud data warehouses Experience implementing data quality, monitoring, and alerting solutions Preferred Skills Scala or Java development experience Databricks experience Docker and Kubernetes CI/CD and DevOps practices Experience supporting ML/Data Science workloads Responsibilities Design and develop scalable batch and real-time data pipelines using Spark and Kafka/PubSub Build and optimize BigQuery-based data platforms and lakehouse architectures Develop ETL/ELT frameworks for data ingestion, transformation, and delivery Optimize Spark jobs, SQL queries, and data workflows for performance and cost efficiency Implement data quality, monitoring, validation, and alerting mechanisms Collaborate with Data Scientists, Analysts, and Business Teams to deliver reliable data solutions Support production deployments and troubleshoot complex data engineering issues #J-18808-Ljbffr Socket.dev