Junior Data Engineer

Junior Data Engineer Job Description Template

Our company is looking for a Junior Data Engineer to join our team.

Responsibilities:

  • Assisting in quality control of quantitative and qualitative research projects;
  • Designing, building and support data processing pipelines to transform data using Hadoop technologies;
  • Working with business analysts to understand business requirements and use cases;
  • Developing and executing quality assurance and test scripts;
  • Building survey and voter file data pipelines;
  • Designing, building data assets in HIVE;
  • Designing and developing tools for internal team use to simplify complex task flows;
  • Designing schemas, data models and data architecture for Hadoop and HBase environments;
  • Implementing data flow scripts using Unix / Hive QL / Oozie scripting;
  • Creating scripts and workflows to automate repeated data processing tasks;
  • Working with data team to identify opportunities for task automation and helpful tools.

Requirements:

  • Understanding of best practices for building Data Lake and analytical architecture on Hadoop;
  • Familiarity with MapR distribution of Hadoop;
  • Scripting / programming with UNIX, Java, Python, Scala etc;
  • Experience with Graph database;
  • Experience in working in large environments such as RDBMS, EDW, NoSQL, etc;
  • Knowledge in schema design, developing data models and proven ability to work with complex data;
  • Understanding Hadoop file format and compressions;
  • Hands-on experience with Hadoop, MapReduce, Hive, Oozie, Airflow, ElasticSearch;
  • Knowledge in real time data ingestion into Hadoop;
  • Experience with Test Driven Code Development, SCM tools such as GIT, Jenkins i.