Big Data Engineer

Senior Big Data Platform Development EngineerAs Senior Software Engineer, you will evolve the data cloud platform and build pipelines that are scalable, robust and resilient, to validate, ingest, normalize/enrich and support business rule processing of healthcare data. You will provide leadership in big data and cloud technologies. You will work with engineering, product management, program management and operations teams planning and delivering the data platform products. You are an independent and resourceful backend or full-stack development engineer with advanced design and problem-solving skills, demonstrated competency in building enterprise and cloud products, and you enjoy working in diverse and fast paced engineering teams.Your duties and responsibilities in this job include:Develop a scalable and resilient cloud data platform and scalable data pipelines.Ensure industry best practices around data pipelines, metadata management, data quality, data governance and data privacy.Build highly scalable AWS Infrastructure (from scratch or through 3rd party products) to enable Big Data Processing in the platform.Be a thought leader and partner in the development and execution of the Enterprise Technology Strategy.Create new opportunities in this space for Change with new feature capabilities and new offerings that drive customer and business value.Partner with Product Management and Business leaders to drive Agile delivery of both existing and new offerings.Assist with the Leadership and collaboration with engineering organizations within Change to manage and optimize portfolio.The requirements needed for this position are:5+ years of experience with Big Data pipelines with Spark in Java, Scala, Python.5+ Years of extensive AWS experience in building Enterprise scale applications and services.5+ Years of experience with AWS services like ECS Fargate, EKS, Docker containers, Lambdas, EMR, EC2, SNS/SQS, MKS, S3, RDS etc.In-Depth understanding of Orchestration services and hands-on experience with Airflow.Proven track record of building scaled data platforms and enterprise products, working in large engineering teams.Experience with public Clouds such as AWS, Azure, GCP.Extensive Experience in building Enterprise Level AWS infrastructure using Terrafrom and/or Cloud Formation Templates.Excellent understanding of DevOps and CI/CD concepts and ability to create deployment Pipelines.Ability to work with internal stakeholders like Networking/API Market Place/Infosec etc. and external ones like third party apps to build new services inside the platform.Strong backend programming skills for data processing, with practical knowledge of availability, scalability, clustering, microservices, multi-threaded development and performance patterns.