Data Engineer

Gke Data Engineering LeadOur client is in need of a GKE Data Engineering Lead for a 2 year engagement to assist in solutioning the architecture as we transform the rebate processing system from legacy platform. The potential candidate will be helping the engineers in rearchitecting, transforming, and designing data pipelines.This resource will come in and help do a current state assessment, recommend the framework and lead team in building the solution for the migration.DataProc, Spark, AirFlow, Composer, etc...Required Experience:3 plus years of experience with Apache Airflow or Composer3 plus years of experience with Data Proc or Spark5 – 7 years of experience with CI/CDExperience completing current state assessments for migration projectsExperience with producing architecture solutions with Composer and SparkJava Engineers and Architects will work with these Data Engineers to understand and help build out that environmentThey will be part and parcel of the work they are doing, helping the rest of the team to build out the environment based on the recommendations they put forth in their assessmentsNeeds to be a leader, understands how to assess problems and navigate the team through any issuesMust understand Java - it is what the environment operates on.Must be proactive as they need a leader to help direct the flow of the group's work.Healthcare experience desiredExperience with producing architecture solutions with Composer and SparkExperience in large scale batch processing with cloud services.Desired Skills: Apache Spark, Map Reduce, Spark Streaming, PySpark, Hive, HDFS, Kafka, Redis, Sqoop, Oozie, IBM Data Sage, SpringbatchRequired Skills: CI/CD, API, Java