Sr Data Engineer

Big Data Engineer

Experience Required - 6+ Years Must Have Technical/Functional Skills

Strong hands-on experience in Big Data technologies and distributed data processing frameworks.

Proficiency in Python for data engineering, scripting, and performance optimization.

Extensive experience with Apache Spark (PySpark/Spark SQL) for large-scale data transformation and processing.

Solid experience in designing and developing ETL/ELT pipelines for batch and real-time data processing.

Experience working with Google Cloud Platform (GCP) services such as BigQuery, Cloud Storage, Dataflow, Dataproc, Composer, and Pub/Sub.

Strong understanding of data warehousing concepts, dimensional modeling, and data lake architectures.

Experience with CI/CD pipelines and DevOps practices for data engineering workflows.

Proficiency in version control systems (Git) and build/release management.

Experience with workflow orchestration tools such as Apache Airflow or Cloud Composer.

Experience working in Agile/Scrum environments with cross-functional teams.

Strong analytical and problem-solving skills.

Strong collaboration and stakeholder communication skills.

Roles & Responsibilities

Design, develop, and maintain scalable Big Data pipelines using Spark (PySpark/Spark SQL) for batch and real-time processing.

Build and optimize robust ETL/ELT workflows to ingest, transform, and load data from multiple structured and unstructured sources.

Develop high-performance SQL queries and implement efficient data models to support analytics and reporting needs.

Architect and implement data solutions on Google Cloud Platform (GCP) using services such as BigQuery, Dataproc, Dataflow, Cloud Storage, Pub/Sub, and Composer.

Monitor, troubleshoot, and resolve production issues in distributed and cloud environments.

Automate operational processes and enforce coding standards, version control, and documentation practices.

Collaborate with cross-functional teams including business stakeholders, data engineers, and product teams.

Drive analytics initiatives to provide actionable insights.

Participate in Agile ceremonies including sprint planning, design reviews, and retrospectives.

Continuously improve reporting frameworks and data processes in alignment with Agile delivery practices.