Senior Data Engineer

Job Description

Senior Data Engineer (GCP / PySpark / ETL)
Position Summary

We are seeking a Senior Data Engineer to join a high-performing Data Engineering team supporting enterprise Fraud and Claims Analytics initiatives. This role will be instrumental in modernizing a large-scale data platform by migrating data assets from Teradata to Google Cloud Platform (GCP). The ideal candidate has extensive experience designing and building scalable data pipelines, developing ETL solutions, and working with big data technologies in cloud environments.

This is an excellent opportunity to work on a high-impact migration involving nearly 1 petabyte of enterprise data while collaborating with engineering teams, business stakeholders, and analytics partners.

Key Responsibilities

• Design, develop, and maintain scalable ETL pipelines supporting Fraud and Claims Analytics.
• Build and optimize high-volume data processing solutions using PySpark , Python , and SQL.
• Participate in the migration of enterprise data assets from Teradata to Google Cloud Platform (GCP) .
• Develop data solutions leveraging BigQuery , PySpark , Hadoop , and cloud-native technologies.
• Collaborate with business stakeholders, data architects, and application teams to understand data requirements and deliver reliable solutions.
• Optimize data pipelines for performance, scalability, and cost efficiency.
• Support multiple applications utilized by Fraud Analytics teams.
• Work with large-scale datasets consisting of approximately 1,000 terabytes of data and 600-700 enterprise tables .
• Schedule and monitor production workloads using AutoSys .
• Develop and maintain data integration workflows using Ab Initio .
• Troubleshoot production issues and implement performance improvements.
• Participate in code reviews and follow engineering best practices for data quality, governance, and documentation.
• Support cloud deployment, testing, and production implementation activities.

Required Qualifications

• Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent experience).
• 5+ years of Data Engineering or Software Engineering experience (Senior Engineer level).
• 4-6 years of hands-on experience with:
  • SQL
  • ETL development
  • Python
  • PySpark
  • AutoSys
  • Ab Initio
  • Cloud technologies (Google Cloud Platform preferred)
• Strong understanding of data warehousing concepts and large-scale data processing.
• Experience building highly scalable data pipelines and data integration solutions.
• Strong analytical, troubleshooting, and problem-solving skills.
• Excellent communication and collaboration skills.