Hadoop Resource / Remote, 1 Month Contract

Job Title

Phase 1: Preliminary Setup and Planning:

• Confirm PowerScale, PowerEdge, Operating System, and Connectivity readiness, ensuring all systems are operational and suitable connectivity has been established.
• Validate infrastructure details, including worker nodes, edge nodes, and storage configurations.
• Verify network configurations align with project requirements.

Phase 2: Pilot Migration Preparation:

• Customer to select a test cluster for the pilot (test) migration.
• Establish a dedicated access zone on PowerScale for the pilot cluster.
• Enable native support for HDFS and NFS on PowerScale.
• Set up Hadoop services, including Ambari, Hive, Spark, and other supporting services as required.
• Work with the internal team and document migration processes for the pilot cluster.

Phase 3: Pilot Migration and Evaluation:

• Execute the pilot migration process.
• Monitor and evaluate the success of the pilot migration.
• Gather feedback from the customer and make any necessary adjustments.
• Document lessons learned and improvements for future migrations.
• Conduct Knowledge Transfer (KT) sessions for Circana resources based on the pilot.

Phase 4: Complex Cluster Migration Preparation:

• Identify one of the more complex clusters for migration.
• Establish a dedicated access zone on PowerScale for the complex cluster.
• Enable native support for HDFS and NFS on PowerScale.
• Set up Hadoop services, including Ambari, Hive, Spark, and supporting services.
• Document migration processes for the complex cluster.

Phase 5: Complex Cluster Migration and Finalization:

• Execute the migration process for the complex cluster.
• Monitor and evaluate the success of the complex cluster migration.
• Gather feedback from the customer and make any necessary adjustments.
• Document lessons learned and improvements for future migrations.
• Conduct Knowledge Transfer (KT) sessions for Circana resources based on the complex cluster migration.

Phase 6: Data Migration Challenges and Optimization:

• Address potential challenges in data migration, considering bandwidth limitations.
• Explore creative solutions for optimizing time-consuming and potentially error-prone processes.
• Implement DistCP for data transfers.
• Document data migration processes and optimizations.

Phase 7: Project Closure and Review:

• Project closeout call.
• Confirm the completion of the two migration tasks.
• Conduct a final review with stakeholders to ensure all objectives are met.
• Obtain sign-off from the customer on the successful completion of the migration.
• Document and share the final project as built report, including key metrics and outcomes.
• Customer signs project Customer Acceptance Form (CAF)