Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Principal Applied Scientist CoreAI Foundry Observability

$165.6k - $296.4k
Full-time

Microsoft Corporation

Overview

The CoreAI organization at Microsoft builds the end-to-end Artificial Intelligence (AI) stack and is core to Azure AI innovation and differentiation as well as all of Microsofts flagship products from Foundry GitHub to Office Teams and Xbox. This role is for a Principal Applied Scientist in the Foundry observability group focused on scientific innovations for AI agents and agent-based systems. The work centers on defining how we measure diagnose and improve agent quality across offline evaluation online signals and production telemetry and then turning that science into practical capabilities for Foundry customers.

In this role you will lead the science for agent and multi-agent systems including offline online and continuous evaluation and agent improvement methods tied to real product behavior. You will build measurement frameworks that help explain regressions detect failure modes and support higher-confidence product and release decisions.

Microsofts mission is to empower every person and every organization on the planet to achieve more and were dedicated to this mission across every aspect of our company. Our culture is centered on embracing a growth mindset and encouraging teams and leaders to bring their best each day. Join us and help shape the future of the world.

Responsibilities
  • Develop scientific frameworks for agent lifecycle spanning quality safety reliability cost and behavioral consistency.
  • Define scientifically grounded quality signals and benchmarks for agent systems including task success tool-use effectiveness plan quality failure modes coordination quality and user-perceived outcomes.
  • Advance observability for AI systems through new approaches to trace analysis agent health modeling behavioral clustering anomaly detection and multi-agent coordination analysis.
  • Partner with engineering teams to operationalize evaluation and observability methods in production systems enabling safe iteration through staged rollouts experimentation A/B testing and automated regression detection.
  • Contribute to instrumentation and semantic standards for agent observability helping make agent execution more explainable diagnosable and comparable across systems.
  • Collaborate deeply with product and platform teams across Foundry Azure Monitor and agent runtimes to shape end-to-end experiences for evaluation benchmarking monitoring and investigation.
  • Act as a technical leader by setting scientific direction driving research-informed product decisions mentoring others and raising the technical bar across the organization.

Qualifications

Required Qualifications:

  • Bachelors Degree inComputer Science Computational Linguistics or related field AND 8 years related experience (e.g. statistics predictive analytics research)
    • OR equivalent experience.
    • OR Masters Degree inComputer Science Computational Linguisticsor related field AND 6 years related experience (e.g. statistics predictive analytics research)
    • OR Doctorate inComputer Science Computational Linguistics or related field AND 5 years related experience (e.g. statistics predictive analytics research)

Other Requirements:

Ability to meet Microsoft customer and/or government security screening requirements are required for this role. These requirements include but are not limited to the following specialized security screenings:

  • Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred Qualifications:

  • Experience and familiarity with large language models (LLMs).
  • Experience with and a solid foundation in large distributed systems algorithms and software engineering principles.
  • PhD in Computer Science Machine Learning Data Science or a related field.
  • 5 years of experience in a research/ML engineering or an applied research scientist position ideally with a focus on AI safety.
  • Hands-on experience with deep learning and transformer-based models
  • Excel at problem-solving and analytics with a proactive approach to challenges
  • Thrive in fast-moving environments where priorities shift and definitions evolve.
  • Enjoy taking ownership end-to-end and learning whatever is necessary to get results.
  • Comfortable working independently while thriving in cross-team collaborations.
  • Understand methods for training and fine-tuning LLMs including distillation supervised fine-tuning and policy optimization.

#coreai

#foundry



Applied Sciences IC6 - The typical base pay range for this role across the U.S. is USD $165600 - $296400 per year. There is a different range applicable to specific work locations within the San Francisco Bay area and New York City metropolitan area and the base pay range for this role in those locations is USD $220800 - $331200 per year.

Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
position will be open for a minimum of 5 days with applications accepted on an ongoing basis until the position is filled.

Microsoft is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to age ancestry citizenship color family or medical care leave gender identity or expression genetic information immigration status marital status medical condition national origin physical or mental disability political affiliation protected veteran or military status race ethnicity religion sex (including pregnancy) sexual orientation or any other characteristic protected by applicable local laws regulations and ordinances. If you need assistance with religious accommodations and/or a reasonable accommodation due to a disability during the application process read more about requesting accommodations.


Required Experience:

IC

Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Principal Applied Scientist CoreAI Foundry Observability in Washington DC vacancy
  •  ...believe the next generation of software won't simply automate tasks—it will reason, plan, adapt, and act autonomously. As a Principal Applied Scientist, you'll help invent that future. You will lead the research and development of frontier AI systems that combine... 
    Principal
    Full time
    Work at office
    Immediate start
    Remote work

    uipath

    Washington DC
    9 days ago
  • $142.8k - $274.8k

     ...Overview Microsoft Foundry helps developers and organizations...  ...efficient for customers. As a Principal Software Engineer you will...  ...with automated testing observability reliable recovery and safe releases...  ...or TensorRT-LLM. #coreai Software... 
    Principal
    Ongoing contract
    Full time
    Local area

    Microsoft

    Washington DC
    2 days ago
  • $105k

     ...decisions? Do you enjoy working across applied mathematics, software development,...  ...team at APL! We are seeking a Research Scientist to develop mathematical and computational...  ...under uncertainty. Integrate data, sensor observations, estimates, and existing knowledge from... 
    Suggested
    Temporary work
    For contractors
    Work experience placement
    Interim role
    Relocation package
    Flexible hours

    Johns Hopkins Applied Physics Laboratory

    Laurel, MD
    1 day ago
  •  ...built on a formal mathematical foundation. Our researchers apply formal analysis techniques to the design and verification of...  ...organizational structure, visit Life at Galois. About This Role Principal Scientists drive a research agenda and develop R&D opportunities at the... 
    Principal
    Local area
    Immediate start

    Galois

    Arlington, VA
    2 days ago
  •  ...Job Description Job Description Principal Research Scientist – LC-MS/MS Bioanalysis Everett, Washington | Relocation Assistance Available...  ...available for qualified candidates. Interested? Apply directly or send your résumé to: ****@*****.***... 
    Principal
    Contract work
    Relocation package

    RPM ReSearch

    Washington DC
    27 days ago
  • $113k - $234.6k

     ...unencumbered and will need your contribution to make it a world class engineering center with the focus on excellence. As our Senior Applied Scientist, you will play a key role in shaping the future of AI at Oracle, with an emphasis on Large Language Models (LLMs) and... 
    Temporary work
    Flexible hours

    Oracle

    Washington DC
    3 days ago
  • $195k - $370.5k

     ...approach to accelerated computing. We're looking for a passionate scientist at the intersection of quantum device physics and advanced...  ...help accelerate the entire quantum ecosystem. As a Sr. Quantum Applied Research Scientist, you will help design and build high-fidelity... 
    Full time

    Nvidia

    Washington DC
    8 days ago
  • $165.6k - $296.4k

     ...Overview CoreAI is at the forefront...  ...software.   The Principal Data Science...  ...Product, Engineering, Applied Science, and Data...  ...teams across Foundry, including areas...  ...models, agents, observability, and Responsible...  ...principal-level data scientists, set analytical priorities... 
    Principal
    Ongoing contract
    Full time
    Local area

    Microsoft

    Washington DC
    1 day ago
  • $134.5k - $265.1k

     ...computing? Would you like to be part of the development of a new firm offering? We are currently seeking an experienced Senior Applied Research Scientist - Tensor Networks to help deliver advanced computational solutions for mission-relevant problems. As Senior Research... 

    Deloitte

    Rosslyn, VA
    1 day ago
  •  ...company. And people who care—about each other, about UiPath, and about our larger purpose. Could that be you? As a Senior Applied Scientist, you will lead the architecture, research, and productization of these next-generation ML systems, bridging deep research with... 
    Full time
    Work at office
    Immediate start
    Remote work

    uipath

    Washington DC
    9 days ago
  • $167.1k - $226.1k

     ...Amazon Supply Chain Optimization Technology (SCOT) Optimal Sourcing Systems (OSS) is looking for an Senior Applied Scientist to help build the next generation strategic sourcing systems. The OSS owns the optimization of inventory sourcing and the orchestration of inbound... 
    Full time
    Temporary work
    Worldwide
    Flexible hours

    Amazon

    Washington DC
    6 days ago
  • $137.1k - $201.6k

     ...Strong statistical judgment about randomized experiments and observational estimation, including confounding, treatment overlap, uncertainty...  ...and parents, and veterans are strongly encouraged to apply. Thank you to the Level Playing Field Institute for this statement... 
    Hourly pay
    Work at office
    Local area
    Flexible hours
    Shift work

    DoorDash USA

    Washington DC
    4 days ago
  • $102.1k - $202.2k

     ...Overview Microsoft Foundry helps developers and organizations build, deploy and operate...  ...isolated systems with automated testing, observability, reliable recovery and safe releases....  ...PyTorch, JAX, Slime, verl or TRL. #coreai Software Engineering IC3 - The typical base... 
    Ongoing contract
    Full time
    Local area

    Microsoft

    Washington DC
    3 days ago
  • $142.8k - $193.2k

     ...the globe, and your work will determine how efficiently and effectively that happens. We are looking for a curious, resourceful Applied Scientist to tackle multi-objective optimization problems worldwide. In this role, you will build mathematical models and AI/ML systems... 
    Hourly pay
    Local area
    Worldwide
    Flexible hours
    Day shift

    Amazon

    Arlington, VA
    14 hours ago
  • $142.8k - $193.2k

    Application deadline: Oct 12, 2026Applied Scientists in AWS Science of Security are dedicated to making AWS the best computing service in...  ...skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional... 
    Flexible hours

    Amazon

    Arlington, VA
    2 days ago
  • The Henry M. Jackson Foundation for the Advancement of Military Medicine, Inc. (HJF) seeks a Sr Scientist to serve as the Principal Investigator and lead independent, externally funded research programs in surgery, trauma, and related disciplines within the USU/WRNNMC Department... 
    Principal

    Henry M. Jackson

    Bethesda, MD
    3 days ago
  • $142.8k - $193.2k

     ...This is an Applied Scientist position in the AWS Automated Reasoning this role you will work together with leading scientists in the field to develop and apply the latest automated-reasoning techniques formal methods and neurosymbolic approaches to ensure safety security... 
    Full time
    Flexible hours

    Amazon

    Washington DC
    6 days ago
  • $5,690 - $11,210 per month

     .... This opportunity will allow you to bring your aspirations talent potential and excitement for the journey ahead. As an Applied Scientist Intern at Microsoft you will be at the forefront of technological innovation working collaboratively to bringresearch to life... 
    Ongoing contract
    Internship
    Local area
    Immediate start

    Microsoft

    Washington DC
    7 days ago
  • $80k - $230k

     ...Engineering is seeking a Research Scientist - Microelectronics to support...  ...Scientist will support applied research focused on the design...  ..., procurement (i.e. domestic foundry interaction) Wafer-Level Design...  ..., ranging from Junior to Principal depending upon education and... 
    Temporary work
    Flexible hours

    TAP Engineering LLC

    College Park, MD
    3 days ago
  • Job Description Job Description Position Summary Our client is seeking a Sr. Scientist to lead development, qualification, and validation of analytical methods supporting characterization, release, and stability testing of biologics and bioconjugate (ADC) products. Key... 
    Principal

    Workforce Genetics

    Beltsville, MD
    3 days ago
  • $104k - $166k

     ...and growth. We bring together forecasters, economists, applied scientists, data scientists, and data visualization engineers to build...  ...models to support prediction and forecasting. Transform observational data into causal models using clever identification Design... 
    Summer work
    Internship
    Summer internship
    Work at office
    Local area
    Relocation

    Zillow

    Washington DC
    14 hours ago
  • $142.8k - $193.2k

    Applied Scientists on the AWS Vulnerability Management (AVM) Builder Enablement team use program analysis, machine learning and generative AI for managing vulnerabilities and securing the software supply chain across AWS. Our scope covers the third-party open-source packages... 
    Flexible hours

    Amazon

    Arlington, VA
    3 days ago
  •  ...About the Role We are hiring a Applied AI Engineer for a leading fintech company's AI Team to serve as the team's technical anchor in the United States. You will lead two primary streams: AI Coding tooling and developer experience — bringing the engineering depth... 
    Principal
    Full time

    ByLabs

    Washington DC
    2 days ago
  • $225k - $275k

     ...Principal Scientist ID 2025-2214 Location US-CA-Menlo Park Practice Polymers and Chemistry Position Type Full-time Posted Salary Range USD $225,000.00 - USD $275,000.00 /Yr. About Exponent Exponent is the only... 
    Principal
    Full time
    Work at office
    Flexible hours

    Exponent

    Bowie, MD
    3 days ago
  •  ...we will change the future of energy, because the world can't wait. What You Will Be Doing: Helion is hiring a Principal Applied Plasma Data Scientist specializing in physics‑grounded machine learning to accelerate experimental FRC plasma modeling. This role will... 
    Principal
    Full time
    Temporary work
    Work at office

    helion

    Washington DC
    9 days ago
  • $24 - $31 per hour

     ...50,000 planners, designers, engineers, scientists, digital innovators, program and construction...  ...measures in natural systems. Apply standard field practices and techniques...  ...hydrology, and mathematics) to explain observations. Prepare preliminary data and scientific... 
    Hourly pay
    For subcontractor
    Casual work
    Internship
    Seasonal work
    Work at office
    Local area
    Immediate start
    Remote work
    Worldwide
    Relocation package
    Flexible hours

    AECOM

    Arlington, VA
    2 days ago
  • $200k - $300k

     ...0+ Ph.D.s 90+ Disciplines 30+ Offices globally Our Opportunity We are currently seeking an accomplished leader for a Principal Scientist (Toxicology) position in our Health Sciences Practice in our Washington, DC office. In this role, you will work as part of a... 
    Principal
    Work at office
    Flexible hours

    Exponent

    Washington DC
    14 hours ago
  • $139.1k - $231.9k

     ...changing what’s possible in science, and few places let you apply it to a problem as consequential as human health. Pfizer...  ...develop, and deliver medicines to patients. As a Senior Principal Machine Learning Scientist, you will help create the scientific foundations behind... 
    Principal
    Permanent employment
    H1b
    Local area
    Worldwide
    Visa sponsorship
    Work visa
    Relocation package
    2 days per week

    Pfizer

    Washington DC
    14 days ago
  • $231k

     ...the team's technical direction Mentor senior ML engineers and scientists, providing technical guidance on agentic design patterns, LLM...  ...email domain is @expediagroup.com. The official place to find and apply for roles is . Equal Opportunity Expedia is committed to... 
    Principal
    Worldwide

    Expedia Group

    Washington DC
    14 hours ago
  • $231k

     ...differentiated, and personalized experiences that drive loyalty and traveler satisfaction. We are looking for a Principal Applied Scientist to join our growing Personalization, Search & Recs team. This Principal will be a key contributor to thought leadership,... 
    Principal

    Expedia Group

    Washington DC
    14 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Principal Applied Scientist CoreAI Foundry Observability. Be the first to apply!