Sr. Data Engineer

Sr. Data Engineer

We are seeking a Senior Data Engineer to support the migration of a legacy database environment into a Redshift-based Sustainability Data Platform. This role is focused on the hands-on execution of migration activities involving datasets used to analyze data center sustainability metrics, including water consumption, carbon emissions, and energy usage. The Senior Data Engineer will partner with the existing team to implement complex transformations, troubleshoot migration challenges, and ensure data quality throughout the migration process.

Key Responsibilities

• Design, develop, and implement ETL/ELT pipelines supporting migration activities.
• Build and enhance AWS Glue jobs using PySpark.
• Implement transformations required to populate Redshift target tables.
• Review existing source-to-target mappings and translate legacy logic into target implementations.
• Investigate and resolve complex transformation or data quality issues identified during migration.
• Modify and maintain Airflow (MWAA) workflows as needed.
• Perform detailed data validation and reconciliation between source and target environments.
• Execute testing activities and document migration outcomes.
• Troubleshoot pipeline failures, performance issues, and production defects.
• Collaborate with team members and stakeholders to clarify requirements and resolve technical questions.
• Participate in code reviews and contribute to engineering best practices.
• Support deployment activities and production cutovers.
• Document assumptions, findings, and implementation details throughout the migration effort.

Required Qualifications

• 6+ years of Data Engineering experience.
• Hands-on experience with AWS Glue, PySpark, Amazon Redshift, and Apache Airflow or MWAA.
• Advanced SQL development and troubleshooting skills.
• Experience developing and maintaining production-grade data pipelines.
• Experience supporting data migration initiatives.
• Analytical and problem-solving capabilities.
• Experience performing data validation, reconciliation, and testing activities.
• Familiarity with Git-based development practices.
• Ability to operate independently and navigate ambiguity in evolving environments.

Preferred Qualifications

• Experience with Apache Iceberg.
• Experience migrating legacy databases or data platforms to modern cloud-based data warehouses.
• Familiarity with large-scale operational or infrastructure datasets.
• Experience supporting sustainability, utility, or consumption analytics.
• Exposure to AWS-native analytics ecosystems.

Success Measures

• Complex migration deliverables completed accurately and on schedule.
• High-quality, production-ready pipelines delivered with minimal defects.
• Data validation and reconciliation activities completed successfully.
• Technical issues and transformation challenges resolved efficiently.
• Collaboration with the existing engineering team to meet critical migration milestones.

Everforth Apex is a world-class IT services company that serves thousands of clients across the globe. When you join Everforth Apex, you become part of a team that values innovation, collaboration, and continuous learning. We offer quality career resources, training, certifications, development opportunities, and a comprehensive benefits package. Our commitment to excellence is reflected in many awards, including ClearlyRateds Best of Staffing® in Talent Satisfaction in the United States and Great Place to Work® in the United Kingdom and Mexico.