Lead Cloudera/Hadoop Migration ( JN -072026-139883 )

ROPES AI _ TORC TEST !

Job Description

I am looking for someone who can not only develop code, but also manage communication, handle working with stakeholders like Infrastructure, content team and be able to understand the scope of migration.

Overview: Cotiviti is a leading solutions and analytics company that leverages unparalleled clinical and financial datasets to deliver deep insight into the performance of the healthcare system. These insights uncover new opportunities for healthcare organizations to collaborate to improve their financial performance, reduce inefficiency, and improve healthcare quality. The Senior Software Engineer is responsible for leading design and development of software applications and reusable software components for the Cotiviti Inc product suite, in compliance with predefined coding standards and technical design. This person will collaborate effectively and work with Senior Developers, QA, Product Owners, Project Management, and other stake holders. Cotiviti develops highly innovative applications in healthcare analytics to generate performance, improvement opportunities and value for our clients. Responsibilities: Design and develop high quality, maintainable software modules for the Cotiviti, product suite Conduct unit and integration testing using appropriate methodology and techniques Analyze requirements and specifications and create detailed designs for implementation Analyze and resolve software related issues originated from internal or external customers Continuously update professional knowledge of new technologies as selected and integrated into the Cotiviti, Inc. product suite Change control Enter, transcribe, record, store and maintain information in both written and electronic form. Review software engineering approach to proposed solutions to ensure adherence to best practice Mentor team members in proper software engineering methods and techniques Complete all responsibilities as outlined in the annual performance review and/or goal setting. Required Complete all special projects and other duties as Required Must be able to perform duties with or without reasonable accomodation Required This job description is intended to describe the general nature and level of work being performed and is not to be construed as an exhaustive list of responsibilities, duties and skills required. This job description does not constitute an employment agreement and is subject to change as the needs of Cotiviti and requirements of the job change. Qualifications: Bachelor's Degree in Computer Science, Information Technology, Information Systems, or related field. 10+ years of experience with enterprise and application design patterns that are applied on Hadoop and Big data platforms in aws. The ideal candidate will possess deep expertise in both on-premises Hadoop ecosystems and modern cloud-native big data solutions on AWS. Hands-on experience with Cloudera Hadoop stack (HDFS, Hive, Spark, Oozie, Python) Proficiency in writing, maintaining, and troubleshooting shell scripts (Bash or similar) for automation and operational support. Migration to AWS will need Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines. Develop a comprehensive migration strategy to transfer data, processing logic, and applications to AWS services. Design target architecture using AWS big data services such as Amazon EMR, Airflow, Glue, Lambda. Evaluate and select appropriate AWS tools for data ingestion, storage, processing, and analytics. Oversee the migration process, including data transfer, application re-platforming, and validation. Ensure the new architecture meets performance, scalability, reliability, and security requirements. Collaborate with developers, data engineers, DevOps, and stakeholders to ensure a seamless migration. Create detailed technical documentation and migration runbooks. Mentor team members on AWS big data best practices. Strong understanding of data security, compliance, and governance in the cloud. Proven track record of migrating or modernizing big data workloads to AWS. Experience with cloud automation and Infrastructure as Code (e.g. Terraform). AWS certifications (e.g., AWS Certified Solutions Architect, AWS Certified Big Data Specialty) are highly desirable. Additional skills required Familiarity with xml/json data formats Knowledge of monitoring and logging tools 5+ years of experience with platform engineering tools such as Bitbucket, Gradle, Octopus, Artifactory. Preferred Qualifications Bachelor's degree in Computer Science, Engineering, or related field (often required) Knowledge of healthcare data such as claims and entities such as providers, members and their relationship within the framework of a claim.

Job Responsibilities

• Design and develop scalable, maintainable software and data-processing solutions.
• Analyze existing Cloudera Hadoop architecture, workflows, and data pipelines.
• Develop and execute migration strategies from on-premises Hadoop to AWS.
• Design AWS architectures using EMR, Glue, Airflow, Lambda, and related services.
• Oversee data transfer, application re-platforming, testing, and validation.
• Ensure solutions meet performance, scalability, security, and reliability requirements.
• Troubleshoot software, platform, and data-pipeline issues.
• Create technical documentation, migration plans, and operational runbooks.
• Collaborate with engineering, DevOps, QA, product, and business stakeholders.
• Mentor team members on software engineering and AWS big data practices.