Senior Data Engineer

Overview

If you love the pursuit of excellence and are inspired by the challenges that come through driving innovations that impact how the world lives, works and plays, then we invite you to learn more about Microsoft Business Operations (MBO) - and the value we deliver across Microsoft and to our customers and partners. We offer unique opportunities to work on interesting global projects in an environment that appreciates diversity, focuses on talent development and recognizes and rewards great work.

As a Senior Data Engineer, you’ll build and operate the platforms that power how Microsoft does business—solving complex, high-impact problems at global scale and shipping technology used every day across the company.

Position Description:

As a Senior Data Engineer, you will design, build, and operate software that helps teams launch and manage products across Microsoft. You will work closely with partner engineers, product managers, and stakeholders to deliver reliable, secure, and scalable services, translating customer and business needs into high-quality engineering solutions.

In this role, you will drive engineering excellence through strong design skills, clean implementation, and a focus on operational quality. You will contribute to architecture and technical direction, review code and designs, and use AI-assisted development tools (e.g., GitHub Copilot, agentic coding workflows, GenAI-based code review and test generation) in a disciplined, production-grade way — taking full responsibility for the quality and safety of AI-generated artifacts and applying Responsible AI practices. You will improve developer velocity through automation, tooling, and AI-powered workflows, use telemetry to troubleshoot issues and continuously improve performance, reliability, and cost, and — where applicable — help integrate AI capabilities such as LLMs, agents, and model-backed features into production systems. You will also mentor others and help raise the team's engineering bar through collaboration, experimentation with new AI practices, and a growth mindset.

Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.

Responsibilities

As a Senior Data Engineer, you will own the end-to-end engineering lifecycle for key components and services—designing, coding, testing, deploying, and operating solutions that are secure, reliable, and maintainable.

• **AI-native development: ** Use AI tools across the full SDLC in a disciplined way. Own the quality of AI-generated requirements, designs, and code — yours and your teammates' — and apply Responsible AI practices.

• Data for AI: Build the training, feature, retrieval, grounding, and evaluation datasets that LLMs, and agents depend on. Partner with PM and engineering to make data contracts, freshness, drift signals, and offline/online consistency first-class concerns.

• **Design and architecture: ** Lead design discussions for your project area, evaluate tradeoffs across batch vs. streaming, warehouse vs. lakehouse, ELT vs. ETL, and storage choices for analytical, feature, and vector workloads. Own architectural decisions with minimal oversight.

• Customer requirements: Partner with PM and engineering to define data requirements; ensure feedback loops on data quality, usage, model performance, and downstream product impact are in place.

• High-quality code: Write extensible, secure, performant code for pipelines, transformations, and supporting services. Apply modern patterns including GenAI-assisted development. Drive code reviews and best practices at the product level.

• Testing and quality: Own the test strategy for your area, including data contract tests, schema validation, freshness checks, distribution and drift monitoring, and offline/online parity. Improve the test suite and use AI tools for test automation.

• Dependencies and coordination: Identify cross-team data dependencies, manage upstream producer and downstream consumer impact, and resolve conflicts when semantics or schemas change in ways that affect models or downstream products.

• Planning: Drive your workgroup's project and release plans. Break work into a roadmap including backfills, migrations, and model-impacting changes, and coach others on estimation.

• **Experimentation: ** Design and run experiments — A/B tests, shadow pipelines, offline replays, evaluation harnesses — and interpret results to guide ship decisions for data and for the models that depend on it.

• Deployment and velocity: Drive deployment automation toward zero-touch; strengthen CI/CD for data systems, including reversible migrations, safe backfills, and controlled rollouts of semantic changes.

• Live site: Participate in on-call rotation as a DRI for data pipelines and serving paths. Use telemetry to diagnose, mitigate, and lead retrospectives. Drive metrics that improve reliability, data quality, and customer impact — including model and agent behavior traced back to data.

• Security, privacy, accessibility: Apply security-as-code, threat modeling, and breach-drill practices. Ensure AI safety controls and data governance for the AI features your data supports. Meet privacy, compliance, and accessibility standards.

• Leadership: Lead by example. Mentor engineers on data engineering craft and AI fluency, raise the team's bar, and foster an inclusive culture.

Qualifications

Required Qualifications:

• Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 3+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 4+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.

Preferred Qualifications:

• Master's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 6+ years experience in business analytics, data science, software development, data modeling, or data engineering OR Bachelor's Degree in Computer Science, Math, Software Engineering, Computer Engineering, or related field AND 8+ years experience in business analytics, data science, software development, data modeling, or data engineering OR equivalent experience.

• 2+ years experience with data governance, data compliance and/or data security.

• 4+ years of hands-on software development experience in one or more general purpose programming languages (e.g., C#, Java, C++, Python, JavaScript/TypeScript).

• Hands-on experience designing and operating production data pipelines and platforms at scale.

• Working understanding of how GenAI systems consume data — training datasets, features and labels, embeddings, retrieval and grounding data, evaluation harnesses, and the data-side failure modes that drive model and agent regressions.

• Hands-on experience using AI-assisted development tools (e.g., GitHub Copilot, agentic coding workflows, GenAI-based code review and test generation) in a disciplined, production-grade way.

• Experience integrating AI capabilities (LLMs, agents, model-backed features) into production systems, including familiarity with Responsible AI principles and applying AI safety controls in production.

• Experience owning a feature area end-to-end, from design through deployment, monitoring, and on-call ownership.

• Experience designing and operating large-scale distributed data systems in a cloud environment (e.g., Azure), including data model and pipeline design, performance tuning, and cost optimization.

• Engineering fundamentals: data structures and algorithms, object-oriented and systems design, and building resilient services (reliability, availability, scalability).

• Experience with DevOps practices and tooling (CI/CD, infrastructure as code, monitoring and alerting, incident response) and a track record of driving toward zero-touch deployment.

• Experience building observable systems: designing telemetry, metrics, and dashboards — including data quality and drift signals — that drive reliability, performance, and customer-impact decisions.

• Experience building secure software, including secure coding practices, threat modeling, premortems, and privacy/compliance considerations relevant to data systems.

• Demonstrated technical leadership through design reviews, mentoring, and driving improvements to code quality and engineering processes.

• Experience collaborating in cross-functional and communicating technical concepts clearly to engineering, product, and executive audiences.

Data Engineering IC4 - The typical base pay range for this role across the U.S. is USD $119,800.00 - $234,700.00 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $160,200.00 - $261,000.00 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:

This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations. (