Senior Data Engineer

Must have : starburst or dremio exp

Role : GCP Data Engineer

Contract

Charlotte, NC - Onsite

Job Description:

Required Qualifications & Skills

Core GCP: Hands-on experience with core gcp data services: Big Query, Cloud Dataflow, Cloud Composer (Airflow), and Cloud Storage.

Databases: Strong proficiency in NoSQL databases (specifically MongoDB) and data warehousing concepts.

Programming: Advanced programming and scripting skills in Python and complex SQL.

Support & CI/CD: Experience utilizing Cloud Logging/Monitoring for root cause analysis and implementing CI/CD frameworks.

Key Responsibilities

Pipeline Integration: Design, build, and deploy scalable ETL/ELT data pipelines using Python and GCP-native services.

Database Management: Integrate data from MongoDB into Google Cloud Storage and load/transform it in Big Query for analytics.

Support & Debugging: Monitor, troubleshoot, and resolve pipeline failures and data discrepancies.

Performance Tuning: Optimize Big Query queries, Dataflow jobs, and MongoDB indexing to ensure cost-efficiency and fast execution.