Data Engineer

We are looking for a Senior Data Engineer with strong experience in AWS Big Data, Hadoop, Spark/PySpark, Kafka, SQL, Shell Scripting, and Scala/Java/Python to support an Enterprise Data Lake.

Purpose

• Senior Data Engineer role to be part of our scrum teams and perform functional system development for Hadoop applications for our Enterprise Data Lake initiative
• This is high visibility fast paced key initiative will integrate data across internal and external sources provide analytical insights and integrate with our critical systems

Essential Responsibilities

• Participate in the agile development process
• Develop functional and technical specifications from business requirements for the commercial platform
• Ensure application quality and adherence to performance requirements
• Help create project estimates and plans Represent engineering team in project meetings and solution discussions
• Participate code review process
• Work with team members to achieve business results in a fast paced and quickly changing environment
• Pair up with data engineers to develop cutting edge Analytic applications leveraging AWS Big Data technologies Kafka PySparkSpark Hadoop Shell Scripting SQL
• Mentor and influence up and down the chain of command
• Perform other duties andor projects as assigned

Qualifications

Requirements

• Bachelors degree in a quantitative field such as Engineering Computer Science Statistics Econometrics and a minimum of 9 years of experience
• Minimum 5 years experience working with and developing big data solutions
• Handson experience on AWS writing shell scripts complex SQL queries Hadoop commands Kafka Spark and Git
• Experience with real time data streaming systems
• Ability to write abstracted reusable code components
• Programming experience in at least two of the following languages Scala Java or Python

Desired Characteristics

• Strong business acumen
• Critical Thinking and Creativity
• Performance tuning experience
• Willingness to learn new technologies quickly
• Superior oral and written communication skills as well as the willingness to collaborate across teams of internal and external technical staff business analysts software support and operations staff
• Strong business acumen including a broad understanding of Synchrony Financial business processes and practices