Data Engineer

Data Engineer

This position can sit anywhere on the East Coast

This position is responsible for design, development and management of data workflows and pipelines in our Integrated Data and Analytics environment. The role is a blend of technical leadership, team leadership, and hands-on development.

Responsibilities/Tasks:

• Responsible for collecting, moving, storing, and preparing data for analytics enablement.
• Participates in project planning sessions, working closely with business analysts and team members to analyze requirements and provides design recommendations for complex systems.
• Designs and develops new data pipelines for ingesting high-volume data sets.
• Develops pipelines using disciplined processes, adhering to industry standards and best practices, including complete ease with agile development methods throughout the SDLC.
• Demonstrates processes to all stakeholders.
• Shares knowledge and provide technical guidance to other team members within area of expertise.
• Takes part in reviews of work, e.g. design and code reviews.
• Works with team members to analyze and understand problems with complex systems and resolve them.
• Resolves customer complaints and respond with suggestions for improvements and enhancements.

This position description identifies the responsibilities and tasks typically associated with the performance of the position. Other relevant essential functions may be required.

Employment Qualifications:

Education:

Bachelor’s degree in a related field. Relevant combination of education and experience may be considered in lieu of degree. Continuous learning, as defined by the Company’s learning philosophy, is required. Certification or progress toward certification is highly preferred and encouraged.

Experience:

2 years’ experience in application development. At least five years of experience in Azure big data services/technologies (Spark, Scala, Azure Data Factory, Azure Data Bricks, Synapse Analytics).

Skills/Knowledge/Abilities (SKA) Required:

• Deep knowledge of SQL/TSQL and Python/PySpark
• Strong understanding of Azure Big Data Services such as Azure Data Lake Storage, Azure Data Factory, Logic Apps, Azure Functions
• Knowledge of AWS cloud and AWS services such as S3 buckets, Lambda, API Gateway
• Experience with Databricks and Delta Lake
• Experience with RDBMS (Oracle 10g/11g, MS SQL), including database design, developing stored procedures and functions, and performance tuning. Working experiences with No-SQL databases.
• Strong background in traditional ETL tools (SSIS, Informatica)
• Understanding of Reporting and Data Visualization tools (PowerBI, Tableau, SSRS)
• Agile software process
• Strong coding skills (Python, R, Scala, Java, C++, C#)
• Working experiences of test-driven development framework.
• Excellent inter-personal and communication skills.
• Ability to multi-task in a fast-paced environment.
• Flexibility to adapt to changing business needs.
• Team player who is delivery-oriented and takes responsibility for the team’s success.
• Enthusiastic, can-do attitude with the drive to continually learn and improve.
• Highly self-motivated, creative and able to dive deep in problem solving.