GCP Data Engineer

GCP Data Engineer

Onsite – Phoenix, AZ

6 months

$50.00 - $52.00/Hour on W2 (Without Benefits). Applicants must be willing to work on W2

Role Description

• In-depth knowledge of the Hadoop ecosystem, including HDFS, MapReduce, Spark, Scala, Hive. Expertise in Hive for data warehousing and querying large datasets.
• Strong experience with Apache Spark for distributed data processing and real-time analytics. Advanced proficiency in Scala programming for big data application development.
• Familiarity with data modeling, ETL processes, and data integration techniques.
• Experience with cloud platforms and big data tools (e.g., Google Big Query).
• Strong analytical and problem-solving skills to address complex data challenges.
• Basic knowledge on JAVA, Spring boot. Strong SQL skill.

Roles Responsibilities

• Architect and design scalable big data solutions leveraging technologies like Hadoop, Hive, Spark, and Scala.
• Develop and optimize data pipelines and workflows for processing and analyzing large-scale datasets.
• Manage and fine-tune distributed computing frameworks to ensure high performance and reliability.
• Collaborate with cross-functional teams, including data engineers, analysts, and business stakeholders, to gather and define data requirements.
• Implement data security, governance, and compliance measures in big data environments.
• Diagnose and resolve technical issues in big data systems to maintain operational efficiency.

Skills Required

• 5+ years of hands-on software development experience with Big Data & Analytics solutions
• GCP Cloud - Big Query, Airflow, DataProc, PubSub, Hadoop Hive, Spark, Python, shell scripting
• Strong experience in designing, developing, and optimizing data pipelines for large-scale data processing, transformation, and analysis using Big Data and GCP technologies.
• Proficiency in SQL and database systems, with experience in designing and optimizing data models for performance and scalability.
• Experience of building event processing pipelines with Kafka or GCP PubSub.
• Design and development experience with Airflow, PubSub Kafka, Git, Jenkins is desirable.
• Knowledge of distributed (multi-tiered) systems, algorithms & relational databases.
• Strong Object-Oriented Programming skills and design patterns.
• Experience with CICD pipelines, Automated test frameworks, and source code management tools (XLR, Jenkins, Git).
• Good knowledge and experience with configuration management tools like GitHub
• Ability to analyze complex data engineering problems, propose effective solutions, and implement them effectively.
• Ability and willingness to learn, adopt and build solutions using the Enterprise frameworks established for Big Data
• Looks proactively beyond the obvious for continuous improvement opportunities.
• Communicate effectively with product and cross functional team.
• Willingness to learn new technologies and leverage them to their optimal potential.
• Understanding of various SDLC methodologies, familiarity with Agile & scrum ceremonies.
• Certifications in cloud platform (GCP Professional Data Engineer) is a plus.

Overall Experience Required is 5 - 6 Years: Essential Skills

• In-depth knowledge of the Hadoop ecosystem, including HDFS, MapReduce, Spark, Scala, Hive
• Expertise in Hive for data warehousing and querying large datasets.
• Strong experience with Apache Spark for distributed data processing and real-time analytics.
• Advanced proficiency in Scala programming for big data application development.
• Familiarity with data modeling, ETL processes, and data integration techniques.
• Experience with cloud platforms and big data tools (e.g., Google Big Query).
• Strong analytical and problem-solving skills to address complex data challenges.
• Basic knowledge on JAVA, Spring boot
• Strong SQL skill.
• 4 to 7 years in Bigdata and Hadoop Ecosystems