Master Data Engineer Job Description Template
Our company is looking for a Master Data Engineer to join our team.
Responsibilities:
- Handle multiple functions and roles for the projects and Agile teams;
- Leverage reusable code modules to solve problems across the team and organization;
- Design robust systems with an eye on the long-term maintenance and support of the application;
- Define, execute and continuously improve our internal software architecture processes;
- Lead and mentor a team of data engineers (both full-time associates and/or third-party resources);
- Provide business, application and technology consulting in feasibility discussions with technology team members and business partners;
- Lead and develop sustainable data driven solutions with current new generation data technologies to drive our business and technology strategies;
- Be a technology thought leader and strategist.
Requirements:
- Bachelor’s degree;
- 4+ years of experience working with AWS: S3, EMR, or EC2;
- At least 4 years of Data Engineering experience with Big Data Technologies: Apache Spark, Hadoop, or Kafka;
- At least 1 year of experience in Cloud technologies: AWS, Azure, OpenStack, Docker, Ansible, Chef or Terraform;
- At least 3 years of microservices development experience: Python, Java or Scala;
- 2+ years of experience with Linux systems;
- 4+ years of experience working with data consumption patterns in SQL or Python applications;
- 3+ years of experience working with automated build and continuous integration systems;
- 6+ years of experience with Data Architecture and Design;
- At least 2 years of experience building data pipelines, CICD pipelines, and fit for purpose data stores;
- Understanding of Object-Oriented and Functional programming concepts using languages like Python and Java;
- At least 1 year of experience with a Cloud platform (AWS, Google, or Azure);
- Understanding of Test-Driven Development concepts and supportive tools like Cucumber;
- 1+ years’ experience designing, developing, and implementing APIs;
- At least 4 years of experience in Python based software application development.