Data Lake Engineer with Scrum Master

Data Lake Engineer with Scrum Master

Role is 75% Data Lake Engineer / 25% Scrum Master responsibilities. Our client is moving towards being a data driven company. Our client's Technology is refactoring how they collect, store and expose data across the Enterprise using the latest cloud architectures, tools, devops mindset for quicker feedback loop while leveraging Agile principles. Their team is looking to solve these pain points with a Data Lake House solution where they store, govern and provide the right data to the right customer at the right time that will integrate and accel both business and technology analytical users. This solution will be the foundation for our client for the next decade using a Cloud-based platform that can scale and continuous integrate with tools and concepts to provide exponential value.

Must Have:

• 1-2 years in Data Lake / Data Pipeline / Lake House
• 3-5 years of AWS cloud development
• DevOps
• CI/CD (What tools have you used?) Monitoring, Metrics & Observability
• Experience with any one of the following languages - Python, Scala, Javascript or Java
• Frameworks
• Serverless / AWS CDK or SAM Cloud Tools - Stacker / Terraform
• AWS Cloud experience
• Infrastructure as Code (IaC) – AWS CloudFormation (Can you describe two hands-on development projects that involved CloudFormation creation thru code? What CI/CD process was used?)
• Identity Access Management (IAM)
• Cloud storage – AWS S3
• Serverless function – AWS Lambda
• Docker

Scrum Master Responsibilities:

• Facilitate Agile Processes: Guide the team in applying Agile principles and Scrum practices, including sprint planning, daily stand-ups, sprint reviews, and retrospectives, to ensure efficient workflow and continuous improvement.
• Remove Obstacles: Identify and help remove any impediments or obstacles that may hinder the team's productivity, allowing team members to focus on high-priority tasks and achieve sprint goals.
• Coach and Mentor: Provide coaching and support to the Scrum team and stakeholders, promoting collaboration, self-organization, and accountability within the team.
• Promote Communication and Transparency: Encourage open communication within the team and with stakeholders, ensuring that project progress, risks, and challenges are visible and managed effectively.

Make sure features are delivered:

• If not technical enough, developers try to buffer in something. Making sure features are well understood by team.
• Driving team to deliverables, removing impediments.

Client's Job Description (ideal skillset below, not all is required, refer to "Must Haves" above):

Skillset

1. AWS Basics – IAM role and policy, VPC, Route53, CloudWatch, API Gateway, Streaming, Storage, Tagging
2. Cloudformation – hands-on delivery experience
3. Cloudformation tools (SAM, Serverless, Stacker, Terraform)
4. DevOps (Jenkins/GitLab/Code Pipeline/Monitoring & Metrics)
5. AWS Cloud Stack:

• Glue Suite
• Catalog
• Crawler
• ETL
• Glue Registry
• Lake Formation
• S3
• Lambda
• Step Functions

Experience

1. 3-5 years cloud development ( AWS, Azure )
2. 1-2 years in Data Lake / Data Pipeline / Lakehouse
3. 10+ years Object Oriented Language - Python, Groovy, Java
4. Quality Assurance / Testing concepts
5. Cloud based development life cycle. Solution includes infrastructure, cost conscientious.
6. Code Quality tool – Prisma, SonarQube, OWASP Security Scanning, VeraCode
7. Data Governance – Life Cycle, Access Model, Security, Privacy