Associate Data Engineer Job Description Template
Our company is looking for a Associate Data Engineer to join our team.
Responsibilities:
- Partner with multiple stakeholders including partners, new product development, BI engineers to develop scalable standard schemas;
- Codify high-performing SQL for efficient data transformation;
- Support operations by identifying, researching and resolving performance and production issues;
- Responsible for data modeling and schema design that will range across multiple business domains within higher education;
- Work with partners to research and conduct business information flow studies;
- Coordinate work with external teams to ensure a smooth development process;
- Participate in efforts to design, build, and develop rapid Proof-of-Concept (POC) solutions and services;
- Collaborate with Data Architects, Business SMEs and Data Scientists to design and develop end-to-end data pipelines and supporting infrastructure;
- Be a key team member in design and development of the Data pipeline for Engineering team;
- Proactively identify and implement opportunities to automate tasks and develop reusable frameworks;
- Apply knowledge of basic principles, methods and practices to simple and moderately complex assignments;
- Build new infrastructure and analytics tools using Python, SQL and AWS.
Requirements:
- Excellent troubleshooting and analytical skills;
- Bachelor’s Degree in Computer Science or Engineering;
- Experience developing commercial software products;
- Master’s degree in Computer Science or Engineering;
- Proven ability to work with users to define requirements and business issues;
- Experience developing ETL processes;
- Strong oral and written communication skills;
- GIT expertise;
- Experience with Schema Design & Dimensional data modeling;
- Experience with AWS Services like EC2, S3, Redshift/Spectrum, Glue, Athena, RDS, Lambda, and API gateway;
- Proficient in one of the coding languages (Python, Java, Scala);
- Experience building large scale data warehouses and ETL data processing pipelines;
- Ability to learn quickly, be organized and detail oriented;
- Experience with Spark, Hive, Kafka, Kinesis, Spark Streaming, and Airflow;
- Has hands on experience writing SQL using any RDBMS (Redshift, Postgres, MySQL, Teradata, Oracle, etc.).