Data Platform Technical Lead / Senior Data Engineer

Data Platform Technical Lead / Senior Data Engineer

The purpose of this position is to provide technical leadership, guidance to team and work independently as well as to build and integrate robust, fault-tolerant backend services. This includes collaboration in requirements definition, prototyping, design, coding, testing and deployment.

Person in this position would exhibit the following behaviours:

• Provides Technical Leadership and guides the team as needed
• Develops and maintains positive working relationships with others
• Shares ideas and information
• Assists colleagues unprompted
• Takes pride and leads by example in the achievement of team objectives
• Has credibility with peers and senior managers
• Self-motivated driven to achieve results
• Keeps pace with change acquires knowledge/skills as the business evolves
• Handles confidential information with sensitivity
• Constantly improves domain knowledge and applies to the project.

JOB RESPONSIBILITIES

• Should be able to Lead the design and develop complex systems using design pattens and implement software, data solutions by developing documentation, flowcharts, layouts, diagrams, charts. Behaviour driven and Test-Driven development (BDD/TDD) would be preferred.
• Capable to drive and lead the design discussions and actively participate with Technical Product Owners, Scrum Master, Architects, Agile team and Operations team to define plans for each Sprint
• Take complete ownership of Team goals for the Sprint and help the team to reach the sprint goals by providing technical guidance and mentoring team with best practices.
• Data Operations Tasks including loading and troubleshooting data processing
• Development and support of our systems including maintaining cost efficient AWS resource consumption (both processing and data base)
• Own and provide guidance on complex stories and completes them in the appropriate sprint.
• Identifies and resolves dependencies and risks involved in completing the stories upfront and shares proactively.
• Participates proactively in the daily stand ups, Sprint planning and retrospectives.
• Follows git flow (Pull requests) and maintains the unit test coverage as well as passes the quality gate for the code that is written.
• Communicates technical and operational issues to both engineering and leadership in a timely manner.
• Ensure that all technology standards are adhered to while developing systems
• Work collaboratively with Technical Architects, Product owners and QA Team to define and ensure the completeness and accuracy of system / technical requirements.
• Working collaboratively with other teams such as infrastructure teams to fulfil the story requirements
• Need to mentor team members as appropriate and proactively support the team for accomplishing the team deliverables

RELEVANT EXPERIENCE & QUALIFICATIONS:

• Masters or bachelor's in engineering in IT/ Electronics / Communication / Computer Science / Information Systems
• 10+ years of strong hands-on experience in complete Software Development Life Cycle with Python, Apache Airflow, Apache Spark, Any Database and Java
• Experience in working with AWS DevOps infrastructure including CloudFormation, IAM, S3, VPC/Networking, Docker container running in AWS Fargate/Batch
• Should be able to build processes to enable efficient flow of work requests, support and product maintenance
• Should be hands-on with APIs, serverless, microservices architecture and leveraging CI/CD pipelines
• Should be hands-on working with git flow, pull requests and other version control related concepts
• Should be proficient in debugging applications, troubleshooting by running from the IDE as well as remote debugging, basic performance testing & tuning
• Should be able to estimate the effort involved in developing a feature/story
• Should be able to review, guide the other Junior, SEs/SSEs and be able to do code review, integrate the work

Good To Have:

• Knowledge on PostgreSQL in Amazon RDS is desirable
• Minimal knowledge of UX/UI design
• Minimal knowledge of application performance monitoring and observability (i.e. Datadog or New Relic)
• Knowledge of SAFe (Scaled Agile Framework) or any other agile framework

SKILLS & COMPETENCIES:

• Excellent verbal and written communication skills
• Strong organizational skills and adaptability to rapidly changing priorities and workloads
• Ability to work well independently and maintain focus in a highly dynamic work environment