Lead Data Engineer

Big Data exp: Spark, Nifi pipelines, data bricks Java, SQL, and PL/SQL ETL Development (They use SSIS)

Virtual or Onsite Interview (Needs to be available for onsite if needed)

Leads software engineering projects and creates the development and delivery of enhanced software solutions. Researches alternative technical solutions for changing business needs. Responsible for assuring the daily activities of the team meet the requirements of SLAs.

Experience dealing with large volumes of data, from various sources, both structured and unstructured. Ability to triage and talk through performance / scaling issues of dealing with data at scale. Good understanding of how data will be read (file formats, partitioning, bucketing). Extensive experience writing testable jobs using Spark (or equivalent) framework.

Programming & Scripting Languages: Java EE, Scala, Spark, SQL, Bash.

Web services & API standards: REST, OAuth, JSON.

Software Architectures (micro-services, event driven, peer-to-peer). Application Security. Asynchronous Pub-Sub and Point to Point Messaging Systems.

Advantage, if you have experience working in ETL and Hadoop Ecosystem: HBase, Solr, Spark Streaming, Kudu, Spring Boot, Spring Context, Spring Data Rest, General Cloudera experience.

• Successfully lead major projects and complex assignments with broad scope and long-term business implications.
• Create solutions with high level of innovation, cost effectiveness, high quality and faster time to market.
• Lead day-to-day system development and maintenance activities of the team to meet service level agreements (SLAs).
• Partner with Project Managers in implementation of processes per the evolving needs of the team and the solutions.
• Create detailed cost and time estimates for implementing business solutions, define technical scope and requirements of the project and assist project manager with managing scope during the lifecycle of the project.
• Ensure adequate test coverage in Unit Testing, System Testing/Integration Testing and Performance Testing.
• Perform Quality Inspections and Walkthroughs throughout the SDLC including Requirements Review, Architecture Review, Design Review, Code Review and Security Review to ensure compliance.
• Coordinate offshore development and manage day-to-day activities.
• Support collection and reporting of project and software metrics.
• Actively look for opportunities to enhance standards and improve process efficiency.
• Mentor and guide other team members during all phases of the SDLC.
• Lead and guide team through all phases of the SDLC of an application.
• Successfully lead production incident and problem management, identification of root cause and remediation of the problems.
• Mentor, coach and assist project team members.
• Ensure compliance to the defined standards.
• Actively look for opportunities to modify and enhance standards per the needs of the department.
• Develop documentation templates, lead and enforce development of documentation throughout the development lifecycle of the solution.
• Ensure compliance of Audit Requirements by proactively educating team on Compliance Requirements and Integrating Compliance Requirements in SDLC.

Bachelor's degree in information systems, Information Technology, Computer Science, Engineering or equivalent experience Strong experience with SQL and PL/SQL development Experience working with relational databases such as SQL Server, Oracle, Postgres Proficient in building ETL pipelines with ETL tools like SSIS or Informatica Understanding of database design and modeling, performance tuning, and stored procedures Excellent analytical, collaboration and communications skills Critical thinking, logical reasoning and experience gathering and understanding business requirements to build ETL pipelines 5+ years of experience in related field, including experience in delivering secure solutions in Financial Services Sector Advanced knowledge and understanding of Software Engineering Concepts and Methodologies

Experience building dashboards with BI tools like Power BI, Tableau

For applications and inquiries, contact:View email address on us.fitly.work