Big Data Engineer

job summary:
The contractor will be responsible for supporting and maintaining the data pipeline platform that powers performance reporting data staging capabilities. Key responsibilities include:

Manage and maintain Amazon Redshift consumer cluster infrastructure, ensuring platform stability, performance, scalability, and operational reliability.

Design, develop, enhance, and support data pipelines built using PySpark and AWS Glue to ingest, transform, and deliver data across the platform.

Monitor, troubleshoot, and optimize ETL/ELT workflows, data processing jobs, and cloud infrastructure to meet service level expectations.

Perform data analysis, root cause investigation, and data validation activities to identify and resolve data quality, performance, and operational issues.

Collaborate with product, engineering, and business teams to support data integration, reporting, and modernization initiatives.

Implement engineering best practices related to automation, monitoring, performance tuning, and cloud-native data processing.

Support production releases, incident resolution, and continuous improvement of the AWS-based data platform.

 
location: Malvern, Pennsylvania
job type: Contract
salary: $39.38 - 44.38 per hour
work hours: 8am to 5pm
education: Bachelors

responsibilities:

Responsibilities

• The contractor will be responsible for supporting and maintaining the data pipeline platform that powers performance reporting data staging capabilities. Key responsibilities include:
• Manage and maintain Amazon Redshift consumer cluster infrastructure, ensuring platform stability, performance, scalability, and operational reliability.
• Design, develop, enhance, and support data pipelines built using PySpark and AWS Glue to ingest, transform, and deliver data across the platform.
• Monitor, troubleshoot, and optimize ETL/ELT workflows, data processing jobs, and cloud infrastructure to meet service level expectations.
• Perform data analysis, root cause investigation, and data validation activities to identify and resolve data quality, performance, and operational issues.
• Collaborate with product, engineering, and business teams to support data integration, reporting, and modernization initiatives.
• Implement engineering best practices related to automation, monitoring, performance tuning, and cloud-native data processing.
• Support production releases, incident resolution, and continuous improvement of the AWS-based data platform.

qualifications:
AWS Glue, PySpark, Amazon Redshift, SQL, data pipeline development, cloud data engineering, data analysis, troubleshooting, and operational support.

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact View email address on randstadusa.com.


Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).

This posting is open for thirty (30) days.

Any consideration of a background check would be an individualized assessment based on the applicant or employee's specific record and the duties and requirements of the specific job.