Data Engineer

Job Title

Total no of positions across all of these skill sets - 25 Remote will do if office, following Genpact locations are fine with client Atlanta, GA Jacksonville, FL Richardson TX NYC

Skill Area - Technologies/Services - Lead - Senior - Intermediate

AWS/Cloud - EMR[Hive, Pyspark],S3, Athena or any other equivalent cloud working experience. - 3+ years of hands-on experience; expert level - 1-2 years of hands-on experience; complex problem solving - Training and small amount of hands-on

Spark - Spark Batch Processing and some familiarity with Spark Structured Streaming - 3+ years of hands-on experience; expert level - 1-2 years of hands-on experience; complex problem solving - Training and small amount of hands-on

Big Data/Hadoop - Working experience with Hadoop stack dealing huge volumes of data. - 3-4 experience working with Hadoop / EMR or any other equivalent technology to process huge volume of data in a scalable fashion. - 1-2 experience working with Hadoop / EMR or any other equivalent technology to process huge volume of data in a scalable fashion; Preferable - Atleast 1 year of experience working with Hadoop / EMR or any other equivalent technology to process huge volume of data in a scalable fashion; Familiarity

SQL and ETL concepts - SQL development and data transformation; analytical functions. - 5+ years of hands-on experience with SQL and ETL concepts; expert level. - 2-3 years of hands-on experience with SQL and ETL concepts; complex problem solving. - 1-2 years of hands-on experience with SQL and ETL concepts.

Python - Batch scripting, data manipulation, distributable packages. - 5+ years of hands-on experience; expert level - 2-3 years of hands-on experience; complex problem solving - 1-2 years of hands-on experience

Scheduler and Orchestration tools - Apache Airflowor equivalent - 5+ years' experience working with batch orchestration tools; hands-on with Airflow preferable - 2-3 years hands-on experience working with batch orchestration tools; hands-on with Airflow preferable - 1-2 years hands-on experience working with batch orchestration tools; hands-on with Airflow preferable

Version Control - Git Hub, Bit Bucket - 5+ years working with code versioning tools such as GitHub or BitBucket; expert level understanding of repo design and best practices - 2-3 years working with code versioning tools such as GitHub or BitBucket. - 1-2 years working with code versioning tools such as GitHub or BitBucket.

DevOps/CICD - Jenkins, Docker, and Kubernetes - 5+ years working with deployment automation tools such as Jenkins and familiarity with containerization concepts such as Docker and Kubernetes. - 2-3 years working with deployment automation tools such as Jenkins - Familiarity with deployment automation tools such as Jenkins

Data Ingestion - Nifi, Kafka, Database ingestion, File Ingestion, API etc., - 5+ years of hands-on experience designing and building ETL pipelines; expert with data ingest, change data capture, data quality; hand on experience with API development; some exposure to Nifi or Kafka - 2-3 years of hands-on experience designing and building ETL pipelines; experience with data ingest, change data capture, data quality; familiarity with API development; some exposure to Nifi or Kafka - 1-2 years of hands-on experience designing and building ETL pipelines; experience with data ingest, change data capture, data quality; familiarity with API development, Nifi, and Kafka

DWH - Snowflake,SQL, Data Modelling, Data Storage and performance tuning. - 5+ years designing and developing relational database objects; knowledgeable on logical and physical data modelling concepts; some experience with Snowflake - 2-3 years designing and developing relational database objects; knowledgeable on logical and physical data modelling concepts; experience with Snowflake preferred - 1-2 years designing and developing relational database objects; familiar with logical and physical data modelling concepts; experience with Snowflake preferred

Reporting - Tableau & Cognos - 3+ years of experience supporting Tableau or Cognos use cases; familiarity with tools and capabilities - 1+ years' experience supporting Tableau or Cognos use cases; familiarity with tools and capabilities - Familiarity with tools and capabilities

Agile - Atlassian JIRA - Familiari