Senior Technical Lead

Maximum Salary (US): 148000Minimum Salary (US): 78000GCP Cloud: Big Query, Airflow, DataProc, PubSubTechnologies: Hadoop Hive, Spark, Python, Shell ScriptingJob SummaryBS or MS degree in computer science, computer engineering, or other technical discipline, or equivalent work experience.5+ years of hands-on software development experience with Big Data & Analytics solutionsGCP Cloud: Big Query, Airflow, DataProc, PubSubTechnologies: Hadoop Hive, Spark, Python, Shell ScriptingStrong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies.Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.Experience of building event processing pipelines with Kafka or GCP PubSub.Design and development experience with Airflow, PubSub, Kafka, Git, Jenkins is desirable.Knowledge of distributed (multi-tiered) systems, algorithms & relational databases.Strong Object-Oriented Programming skills and design patterns.Experience with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).Good knowledge and experience with configuration management tools like GitHubAbility to analyze complex data engineering problems, propose effective solutions, and implement them effectively.Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big DataLooks proactively beyond the obvious for continuous improvement opportunities.Communicates effectively with product and cross functional team.Willingness to learn new technologies and leverage them to their optimal potential.Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.Certifications in cloud platform (GCP Professional Data Engineer) is a plus.Provide engineering expertise to the team for developing features for Banking including Financial Crimes, Anti-money Laundering platformsImprove engineering practices, including driving resilience, predictability, and end efficiency through automationPartner with product teams to envision and deliver solutions which transform Financial Crimes Compliance using technologyKey ResponsibilitiesKey ResponsibilitiesDesign, develop, and optimize large-scale data pipelines using GCP and Big Data technologies.Build and maintain data processing, transformation, and analytics solutions.Implement event-driven data ingestion and processing using Kafka or PubSub.Develop workflow orchestration solutions using Airflow.Design scalable data models and optimize SQL performance.Contribute to distributed, multi-tier enterprise data platforms.Implement and maintain CI/CD pipelines and automated testing frameworks.Collaborate with product and cross-functional teams to deliver business solutions.Drive engineering excellence through automation, resiliency, and operational efficiency.Support Banking, Financial Crimes, and AML compliance initiatives through technology solutions.Adopt enterprise data engineering frameworks and best practices.Continuously improve platform performance, scalability, and reliability.Learn and apply emerging technologies to enhance data platforms.Skill RequirementsPrimary SkillsGCP (BigQuery, DataProc, PubSub)Big Data EngineeringHadoop HiveApache SparkAirflowPythonSQLData Pipeline Design & OptimizationKafka / Event StreamingDistributed SystemsSecondary SkillsShell ScriptingJenkinsGitGitHubXLRCI/CD AutomationObject-Oriented ProgrammingDesign PatternsData ModelingAgile & ScrumConfiguration ManagementGCP Professional Data Engineer CertificationOther Requirements