GCP Data Engineer

JOB SUMMARY

This role involves collaborating with business, IT, Analyst, and Data Science groups to define data requirements. The position requires designing, developing, deploying, and supporting high-performance data pipelines, modeling data platforms, and ensuring data quality. It also includes developing REST APIs for data exposure and mentoring junior data engineers. Key Responsibilities Collect business requirements by working closely with various business, IT, Analyst, and Data Science groups. Design, develop, deploy, and support high-performance data pipelines, both inbound and outbound. Model the data platform by applying business logic and building objects in the semantic layer. Optimize data pipelines for performance, scalability, and reliability. Implement CI/CD pipelines for continuous deployment and delivery of data products. Ensure the quality of critical data elements, prepare data quality remediation plans, and collaborate to fix quality issues. Document the design and support strategy of data pipelines. Capture, store, and socialize data lineage and operational metadata. Troubleshoot and resolve data engineering issues. Develop REST APIs to expose data to other teams. Mentor and guide junior data engineers. Required Qualifications 6+ years of experience in data engineering solutions such as data platforms, ingestion, data management, or publication/analytics. 2+ years of experience in Google Cloud Platform (GCP) with services like BigQuery, Composer, GCS, DataStream, or Dataflows. Expert knowledge of SQL and Python programming. Experience working with Airflow as a workflow management tool, including building operators to connect, extract, and ingest data. Experience in tuning queries for performance and scalability. Experience in real-time data ingestion using GCP Pub/Sub, Kafka, Spark, or similar technologies. Proven experience working in an agile environment. Proven experience working in incremental execution through successful launches. Preferred Qualifications None specified in the provided text. Certifications None specified in the provided text. #J-18808-Ljbffr Compunnel