Senior Big Data Engineer - Cloudera Hadoop to AWS Migration

job summary:
We are seeking a senior-level Big Data Engineer with hands-on experience migrating enterprise Cloudera Hadoop workloads to AWS. The ideal candidate has deep expertise in HDFS, Hive, Spark, Oozie, Python, shell scripting, and AWS services including EMR, Glue, Airflow, and Lambda. This role will lead migration planning, target architecture design, application replatforming, validation, and technical documentation.

 
location: Telecommute
job type: Contract
salary: $75 - 80 per hour
work hours: 9am to 5pm
education: Bachelors

responsibilities:

• Design and develop scalable, maintainable software and data-processing solutions.
• Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines.
• Develop and execute migration strategies from on-premises Hadoop to AWS.
• Design AWS architectures using EMR, Glue, Airflow, Lambda, and related services.
• Oversee data transfer, application re-platforming, testing, and validation.
• Ensure solutions meet performance, scalability, security, and reliability requirements.
• Troubleshoot software, platform, and data-pipeline issues.
• Create technical documentation, migration plans, and operational runbooks.
• Collaborate with engineering, DevOps, QA, product, and business stakeholders.
• Mentor team members on software engineering and AWS big data practices.

qualifications:
Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.

10+ years of experience with enterprise application design, Hadoop, and big data platforms.

Hands-on experience with Cloudera Hadoop, HDFS, Hive, Spark, Oozie, and Python.

Proven experience migrating or modernizing Hadoop workloads on AWS.

Strong experience with Amazon EMR, AWS Glue, Airflow, Lambda, and cloud data architecture.

Proficiency with Bash or similar shell scripting languages.

Experience with Terraform or other Infrastructure-as-Code tools.

Strong understanding of cloud security, governance, compliance, and data protection.

Experience with XML and JSON data formats.

Familiarity with monitoring and logging tools.

5+ years of experience with tools such as Bitbucket and Artifactory.

Strong communication, documentation, troubleshooting, and mentoring skills.

Preferred Qualifications:

AWS certification, such as AWS Certified Solutions Architect or a related data certification.

Experience working with healthcare claims data, providers, members, and related healthcare entities.

Previous experience leading large-scale Hadoop-to-AWS migration initiatives.

Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.

At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact View email address on randstadusa.com.


Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility).

This posting is open for thirty (30) days.

Any consideration of a background check would be an individualized assessment based on the applicant or employee's specific record and the duties and requirements of the specific job.