Aws data engineer

job summary:
Design, build, and maintain scalable data pipelines using Python, PySpark, SQL, and AWS services to support batch and real-time processing needs.

Develop and optimize ETL/ELT workflows for efficient data ingestion, transformation, and loading across cloud platforms.

Translate business and functional requirements into technical designs, SQL queries, and pipeline logic in collaboration with Data Analysts and stakeholders.

Develop, test, and maintain SQL scripts, Python programs, and data processing jobs following coding standards and best practices. Implement reusable, modular, and high-performance data processing components to ensure scalability and maintainability.

Apply data modeling and transformation techniques to structure data for analytics, reporting, and downstream systems. Implement data quality checks, validation rules, and monitoring to ensure data accuracy, consistency, and reliability.

Monitor, troubleshoot, and resolve issues across data pipelines, including ingestion, transformation, and downstream consumption.

Participate in architecture, design, code, and test reviews to ensure quality, compliance, and adherence to enterprise standards.

Follow software engineering best practices, including version control, CI/CD integration, and infrastructure-as-code.

Collaborate with cross-functional teams and communicate technical designs and trade-offs effectively to business and engineering stakeholders.

Identify opportunities to improve data platform performance, efficiency, and scalability, and perform other related duties as assigned.

location: Malvern, Pennsylvania
job type: Contract
salary: $58.33 - 63.33 per hour
work hours: 8am to 5pm
education: Bachelors


responsibilities:

• Design, build, and maintain scalable data pipelines using Python, PySpark, SQL, and AWS services to support batch and real-time processing needs.
• Develop and optimize ETL/ELT workflows for efficient data ingestion, transformation, and loading across cloud platforms.
• Translate business and functional requirements into technical designs, SQL queries, and pipeline logic in collaboration with Data Analysts and stakeholders.
• Develop, test, and maintain SQL scripts, Python programs, and data processing jobs following coding standards and best practices. Implement reusable, modular, and high-performance data processing components to ensure scalability and maintainability.
• Apply data modeling and transformation techniques to structure data for analytics, reporting, and downstream systems. Implement data quality checks, validation rules, and monitoring to ensure data accuracy, consistency, and reliability.
• Monitor, troubleshoot, and resolve issues across data pipelines, including ingestion, transformation, and downstream consumption.
• Participate in architecture, design, code, and test reviews to ensure quality, compliance, and adherence to enterprise standards.
• Follow software engineering best practices, including version control, CI/CD integration, and infrastructure-as-code.
• Collaborate with cross-functional teams and communicate technical designs and trade-offs effectively to business and engineering stakeholders.
• Identify opportunities to improve data platform performance, efficiency, and scalability, and perform other related duties as assigned.

qualifications:
Robust proficiency in Python and PySpark for building scalable data pipelines.

Advanced SQL skills (complex queries, transformations, performance tuning).

Hands-on experience with AWS services: S3, Glue(Job & Catalog), Lambda, IAM, Step Functions, CloudFormation, etc.

Experience designing and implementing ETL/ELT pipelines in AWS cloud environments.

Knowledge of data modeling and structuring data for analytics and reporting.

Experience working with large-scale structured and semi-structured datasets.

Experience implementing data quality checks, validation, and monitoring frameworks.

Robust understanding of software development lifecycle (SDLC) practices.

Experience with version control (Git) and CI/CD pipelines. Familiarity with infrastructure-as-code (CloudFormation or Terraform).

Ability to troubleshoot and optimize data pipelines for performance and cost.

Experience collaborating with stakeholders and translating business requirements into technical solutions.

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact View email address on click.appcast.io.


Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).


This posting is open for thirty (30) days.

Any consideration of a background check would be an individualized assessment based on the applicant or employee's specific record and the duties and requirements of the specific job.