Hadoop Administrator

Hadoop Administrator

Job Location: Plano, TX

Duration: 8 months contract

Preferred:

Cloudera Hadoop administration certification, AWS is plus

Required Skill Set:

• Responsible for implementation and ongoing administration of Hadoop infrastructure.
• Aligning with the platform engineering team to propose and deploy new hardware and software environments required for Hadoop and to expand existing environments.
• Working with application teams to setup new Hadoop users. This job includes setting up Linux users, User group creation, security, testing HDFS, Hive, MapReduce, access for the new users.
• Cluster maintenance as well as creation and removal of nodes using tools like Cloudera Manager Enterprise, AWS and other tools.
• Performance tuning of Hadoop clusters, Kafka Clusters, Hadoop MapReduce routines, Hbase, Impala queries and Scala/Java Spark routines.
• Screen Hadoop cluster job performances and capacity planning.
• Monitor Hadoop and Kafka cluster connectivity and security.
• Manage and review Hadoop log files.
• File system management and monitoring.
• HDFS support and maintenance.
• Diligently teaming with the infrastructure, network, database, application and business intelligence teams to guarantee high data quality and availability.
• Install operating system and Hadoop updates, patches, version upgrades when required.
• Be able to troubleshoot issues with hive, Hbase, Kafka, spark /scala scripts to isolate/fix issues if infrastructure related.
• Be able to tune spark programs with appropriate parameters.
• Minimum 10+ Years of experience is required.
• General operational expertise - good Linux troubleshooting skills, understanding of system's capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks.
• Hadoop skills like HBase, Hive, Sqoop, Impala, Spark, Kafka, Parquet etc.
• Should be able to deploy Hadoop cluster, add and remove nodes, keep track of jobs, monitor critical parts of the cluster, configure name-node high availability, schedule and configure it and take backups.
• Good knowledge of Linux.
• Familiarity with open source configuration management and deployment tools and Linux scripting.
• Desirable: Knowledge of Troubleshooting Core Java Applications