Applied Scientist, LLM Quality & Auditing
Amazon Locker
Amazon Data Services, Inc. seeks an Applied Scientist with a strong ML and statistics background to build auditing systems for data quality. You will lead quality strategies, design framework SOPs, and develop automated evaluation for LLM-based judging.
Collaboration with scientists, engineers, and vendor teams is essential to improve Nova benchmark performance. The role involves audits, coaching, and implementing data-collection quality practices that directly impact customer experiences with
#J-18808-LjbffrVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Applied Scientist, LLM Quality & Auditing in Bellevue, WA vacancy
$136k - $184k
...Applied Scientist, Artificial General Intelligence, Foundational AI Job ID: 10516102 |... ...background in machine learning, statistics, quality assurance, auditing methodologies, and automated... ...involves developing and maintaining LLM-as-a-Judge systems, including designing...Quality16 hoursFlexible hoursAmazon
Bellevue, WA1 day ago$142.8k - $193.2k
...Translations Services (TS) is seeking an Applied Scientist to be based in our Seattle office. As a... ...responsibilities* Apply your expertise in LLM models to design, develop, and implement... ...success metrics, and deliver high-quality solutions.* Conduct thorough data analysis...QualityWork at officeWorldwideFlexible hoursAmazon
Seattle, WA1 day ago$142.8k - $193.2k
As an Applied Scientist in Amazon Fullfilment Technology, you will lead the development of agentic systems to assist with operational... ...; grounding with RAG.- Evaluation: developing decision quality metrics, scaling LLM-based evaluations.About the teamAmazon Fulfillment...QualityFlexible hoursAmazon
Bellevue, WA5 days ago- ...purpose.Could that be you?As a Principal Applied Scientist, you will lead the architecture,... ...deploy advanced ML and AI systems, covering LLM fine-tuning, multimodal pipelines, computer... ...feedback, A/B testing, and cost, latency, and quality analysis.• Serve as a technical leader...QualityWork at officeImmediate startRemote work
UiPath
Bellevue, WA4 days ago $167.1k - $226.1k
...interaction. We are looking for a Senior Applied Scientist to lead the development of generative AI... ...evaluation frameworks that catch quality regressions before customers do and gate... ...stakes, and operates at a scale where naive LLM approaches break downYou'll work across...QualityContract workFlexible hoursAmazon
Bellevue, WA1 day ago$167.1k - $226.1k
...the science of dataset quality — for the data that trains... ...Large language model (LLM)-as-a-Judge is the... ...system that plan its own audits, critiques its own judgments... ...science, mentor junior scientists, and present results to... ...country/region you’re applying in isn’t listed, please...QualityFlexible hoursAmazon
Bellevue, WA3 days ago$218k - $265k
...adapt, and act autonomously. As a Principal Applied Scientist, you’ll help invent that future. You... ...autonomous task completion. Build production-quality AI systems that combine LLMs, multimodal... ...production. Foundation model training LLM post-training Reinforcement Learning...QualityWork at officeImmediate startRemote workUiPath
Bellevue, WA4 days ago$167.1k - $226.1k
...for one. So we can describe quality far better than we can optimize... ...to do. You will distill LLM quality judgments into models... ...signals into online objectives. Scientists on this team own the measurement... ...'s degree and 6+ years of applied research experience- Experience...QualityFlexible hoursAmazon
Seattle, WA2 days ago$167.1k - $226.1k
...SCOT) Optimal Sourcing Systems (OSS) is looking for an Senior Applied Scientist to help build the next generation strategic sourcing systems.... ...coding (e.g., Python) with experience developing production-quality scientific software- Technical depth in one or more of: mathematical...QualityTemporary workWorldwideFlexible hoursAmazon
Bellevue, WA4 days ago$167.1k - $226.1k
We are looking for an seasoned Applied Scientist to design, build, and deploy the ML/AI models that... ...forecasting, anomaly detection, and LLM-based reasoning — all applied to real operational... ...operational data into executive-quality narrative summaries, decision framings,...QualityFlexible hoursAmazon
Seattle, WA5 days ago$119.8k - $234.7k
...than 25%Profession: Research, Applied, & Data SciencesDiscipline: Applied... ...to improve customer value, quality, trust, and adoption across... ...experiences. As a Senior Applied Scientist, you will play a critical role... ...of Large Language Model (LLM), Evaluation, Relevance and Responsible...QualityOngoing contractLocal area3 days per weekMicrosoft
Redmond, WA4 days ago$198.9k - $269k
...subscription. Join our team of Scientists and Engineers developing AI/... ...member value.As a Principal Applied Scientist, you will be the intellectual... ...next-generation Gen AI / LLM-based foundation models that... ...behavioral signals into high-quality latent representations of...QualityFlexible hoursAmazon
Seattle, WA4 days ago$142.8k - $193.2k
...scale. This is one example of an LLM based measurement pipeline you... ...figuring out how to label quality with LLMs, hitting the same problems... ...version; golden sets nobody audited, accuracy that collapses in... ...If the country/region you’re applying in isn’t listed, please contact...QualityFlexible hoursAmazon
Seattle, WA4 days ago$142.8k - $193.2k
...Insights & Optimization is looking for an Applied Scientist to help develop sophisticated algorithms... ...EC2, EBS, Lambda, ECS, RDS, and Aurora, LLM evaluation science for AI-powered agent... ...determine whether capabilities meet the quality bar. We value accuracy over speed,...QualityFlexible hoursAmazonWebServices
Seattle, WA3 days ago$136k - $184k
...predict, and optimize our network operations. We are seeking an Applied Scientist to drive the operational excellence of these models, deeply... ....- Developing evaluation frameworks for models that identify quality regressions before customers do.A day in the lifeYou will be...QualityFlexible hoursAmazon
Bellevue, WA3 days ago$102.1k - $202.2k
...than 25%Profession: Research, Applied, & Data SciencesDiscipline: Applied... ...team is seeking an Applied Scientist to help build the next-... ...grounding and large language model (LLM) training. We operate end-to-... ...that improve system performance, quality, data management, and accuracy...QualityOngoing contractLocal areaMicrosoft
Redmond, WA3 days ago$198.9k - $269k
...measurably faster.We are looking for a Principal Applied Scientist to own the scientific direction of that... ..., or long-horizon context systems for LLM applications- Experience with temporal... ...making the judgment call on when quality is sufficient to ship- Experience with model...QualityLocal areaFlexible hoursDay shiftAmazon
Seattle, WA3 days ago$142.8k - $193.2k
Amazon is looking for an Applied Scientist to help build next generation selection/assortment systems. On the Specialized Selection team within... ...your scientific solutions, including writing production-quality critical path code; Communicating, both written and verbally,...QualityFlexible hoursAmazon
Bellevue, WA5 days ago$172.4k - $223.4k
We are looking for an Applied Scientist II to build the AI behind Auto Optimization: agentic systems... ...on approach, and help raise the quality of the team's science.Key job responsibilities... ...products- Experience designing and deploying LLM-based agents in production.Amazon is an...QualityFlexible hoursShift workAmazon
Seattle, WA4 days ago$142.8k - $193.2k
...Search team's vision is to deliver high quality search results regardless of how customers... ...Large Language Models (LLMs): a new LLM stack that already powers Amazon Search,... ...experiences across Amazon.We are hiring an Applied Scientist to push the science behind this stack:...QualityFlexible hoursAmazon
Seattle, WA2 days ago$167.1k - $226.1k
...founders already use, and we need a scientist to build the ML capabilities... ...deploy them into production.- Apply generative AI and large... ...retrieval, ranking, and evaluation of LLM-powered experiences.- Design... ...experiments that keep model quality quantified and defensible,...QualityWorldwideFlexible hoursShift workDay shiftAmazonWebServices
Seattle, WA4 days ago$119.8k - $234.7k
...ContributorTravel: Less than 25%Profession: Research, Applied, & Data SciencesDiscipline: Applied... ...computer vision, including the latest LLM models. Deploying robust and scalable... ...high level vision and architecture, down to quality design and implementation. Experience...QualityOngoing contractWork experience placementWork at officeLocal areaMicrosoft
Redmond, WA4 days ago$142.8k - $193.2k
...experience feel personal.We're hiring an Applied Scientist to push the boundaries of what's... ...designing retrieval architectures that balance quality with sub-100ms latency constraints.4. Data... ...retrieval, relevance scoring, and LLM-based summarization.2. Fine-tune and evaluate...QualityFlexible hoursAmazon
Seattle, WA1 day ago$142.8k - $193.2k
...Business Trends (CXBT) is looking for an Applied Scientist to help shape the science of evaluating... ...methodologies and metrics that quantify the quality of agent-produced outcomes across... ...Experience with agentic/tool-using systems, LLM-based agents, or the evaluation of...QualityWork experience placementFlexible hoursAmazon
Seattle, WA1 day ago$183.8k - $248.7k
...dedicated, skilled, and innovative Applied Science Manager with a... ...learning, statistics, quality assurance, auditing methodologies, and automated... ...and mentor a team of Applied Scientists who develop comprehensive... ...development and maintenance of LLM-as-a-Judge systems,...QualityFlexible hoursAmazon
Bellevue, WA4 days ago$142.3k - $263.3k
...Sr. Applied Scientist, AI Evaluation & Quality Systems Seattle, Washington, United States Machine Learning and AI Apple Services Engineering (ASE) powers the AI and LLM features behind experiences that hundreds of millions of users love every day. As these systems...QualityRelocationShift workApple Inc.
Seattle, WA8 hours ago$192.2k - $260k
...Applied Scientist III, AWS Account Management and Governance Job ID: 10580997 | Amazon Development... .... Build on and advance a multi-agent LLM framework for autonomous governance... ...organization — raising the bar on science quality, growing other scientists and engineers,...QualityLocal areaFlexible hoursAmazon Inc.
Seattle, WA4 days ago$136k - $184k
...future of shopping.We are seeking a passionate, innovative, and highly skilled Applied Scientist with expertise in AI, Agentic LLMs, Generative AI, Machine Learning, and NLP to help build LLM-powered solutions for Amazon’s BuyForMe product, which enables Amazon...Local areaFlexible hoursAmazon
Seattle, WA3 days ago$112k - $151k
...hospitals. Industrial and physical AI is a major opportunity in applied AI, and one of the harder ones to get right. There is a... ...on Identifyand de-risk scaling challenges in your area: data quality, model drift, latency, throughput, cost, safety Raise the bar...QualityLocal areaSiemens AG
Bellevue, WA2 days ago$183.8k - $248.7k
...advertising.We are looking for a Senior Applied Scientist to build the science that helps Amazon's... ...for each problem, from classical ML to LLM/reasoning methods.- Define and curate the... ...Stay deeply hands-on: write production-quality, critical-path code and build core components...QualityFlexible hoursShift workAmazon
Seattle, WA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Applied Scientist, LLM Quality & Auditing. Be the first to apply!
Related searches
- research scientist machine learning deep learning Bellevue, WA
- health scientist Bellevue, WA
- scientist biology Bellevue, WA
- drug safety scientist Bellevue, WA
- machine learning research scientist Bellevue, WA
- research scientist Bellevue, WA
- applied sports scientist Bellevue, WA
- senior research scientist Bellevue, WA
- manufacturing scientist Bellevue, WA
- application scientist Bellevue, WA