Data Engineer

Big Data Platform ExperienceMinimum 5 years Cosmos (MapReducer), Azure data stack (HDI, ADLS Gen2), Spark/Scala.SQL + High-Level Programming LanguageMinimum 5 years SQL queries plus experience in C#, Scala, Java, or Python (flexible, not a deal breaker).CICD, Debugging & TroubleshootingMinimum 5 years Strong debugging, monitoring, and engineering lifecycle skills.Typical DayWill work with one of the engineers and engineering manager and responsible for feature design, development, validation, CI/CD, bug fixes, troubleshooting & debugging issues Follow engineering practice and deployment rules, SAW device is needed Key Projects • Security work: fixing code security issues, updating identities/secrets, ensuring secure endpoints. • Governance: data partitioning, physical access restrictions, compliance related updates. • Feature innovation and continuous platform enhancements. • Maintaining and improving pipelines across Cosmos (MapReducer) and Azure data platforms (HDI, ADLS Gen2, Spark). Typical Task Breakdown & Operating Rhythm • 15–20% security-related fixes (code vulnerabilities, endpoint issues, secret rotation). • Governance and compliance work (data partitioning, access control). • Writing, modifying, and debugging code and services. • CICD, monitoring, and standard engineering lifecycle practices. • Mostly heads down execution work, following a task list and completing high priority deliverables quickly. • Collaborating with FTEs who will help onboard and accelerate ramp up. • Using SAW device access for AME domain work (pre prod/prod as allowed).Ideal BackgroundDistributed system design & implementation, BIG DATA platforms knowledge, CI/CD processes, debugging & troubleshooting skillsDisqualifiersLong ramp up timeline or lack of experience with large scale data platforms. No experience in high reliability data engineering environments. Inability to work independently with heavy execution demands. Lack of any programming language + SQL exposure (manager expects at least a foundational high level language).Best vs. Average CandidateBest: o Has 5–7 yrs in large scale data engineering (Cosmos, Azure, Spark). o Strong SQL + one high level language (C#, Scala, Java, Python). o Prior Microsoft experience (S360, security expectations, internal processes). o Very fast ramp up and can immediately execute tasks independently.