Tech Lead

Tech Lead

Responsibilities:

• Develop modular and reusable applications using Python.
• Implement configuration-driven design, robust exception handling, logging, debugging, and data processing.
• Utilize BigQuery SQL for complex joins, CTEs, window functions, MERGE operations, and performance tuning.
• Develop production DAGs and manage task orchestration and dependencies using Airflow / Cloud Composer.
• Handle dynamic/runtime configuration, retry strategies, failure handling, backfills, and reruns in production.
• Manage GCP services including BigQuery, Cloud Storage, IAM, service accounts, and Pub/Sub integrations.
• Conduct testing and delivery processes using pytest, unit and integration testing, and Git workflows.
• Implement CI/CD pipelines and validate deployments across environments.
• Develop JSON-driven pipeline architecture and reusable hooks during onboarding.
• Ensure data reliability and financial processing through idempotent reruns, deduplication strategies, and audit checks.

Required Skills:

• Proficiency in Python.
• Experience with BigQuery SQL.
• Familiarity with Airflow / Cloud Composer.
• Knowledge of GCP services and data pipeline operations.
• Strong testing and delivery skills, including CI/CD practices.
• Understanding of data reliability and financial processing principles.