Senior Scientist, AI Quality & LLM Evaluation
Apple Inc.
Apple Inc. is seeking a Staff Applied Scientist in Seattle to lead the data quality validation framework for AI evaluation signals. You will own methodologies, design statistical frameworks for judge reliability, and hands-on build the system that bridges automated evaluation with human ground truth. This is a high-impact IC role emphasizing architectural rigor and practical delivery. You will collaborate with ASE and partner teams to validate new LLM Judge processes, ensuring robust evaluation #J-18808-Ljbffr Apple Inc.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Senior Scientist, AI Quality & LLM Evaluation in Seattle, WA vacancy
$142.3k - $263.3k
Senior Applied Scientist - AI Evaluation & Quality Systems Seattle, Washington, United States Machine Learning and AI Apple Services Engineering (ASE) powers the AI and LLM features behind experiences that hundreds of millions of users love every day. As these systems...QualitySeniorRelocationShift work- Research Scientist, LLM Evaluation & Post-Training page is loaded## Research Scientist, LLM Evaluation & Post-Traininglocations... ...107073**About Centific**Centific is a frontier AI data foundry that curates diverse, high-quality data, using our purpose-built technology...QualityFull timeRemote work
$205.4k - $308.5k
Research Scientist, AI Evaluation Science Seattle, Washington, United States Software... ...levels (early‑career to senior researchers). What unites all... ...reward hacking mitigation); LLM‑as‑judge calibration, rubric... ...active learning, annotation quality, preference elicitation)...QualityRelocation$175k - $263.3k
Measurement Scientist, AI Evaluation Platform Seattle, Washington, United States Software and Services... ...will develop methods for validating LLM‑as‑judge evaluators, automated benchmarks... ...psychometric methods to assessing the quality of benchmarks, for example drawing on...QualityWork experience placementRelocation$136k
...democratize access to high-quality creatives (text, audio,... ..., videos) by building AI-driven solutions for... ...large language models (LLM), generative audio (music... ...-knit team of applied scientists and product managers... ...and explain methods to senior leadership. Willingness...QualitySenior$76k - $125.3k
...that comes with producing high-quality, valued work and advice. Your... ...analysis Compile and evaluate moderately complex data, computations... ...client relationship matters to senior management and subject-matter... ...capital markets. Enabled by data, AI and advanced technology, EY...QualitySeniorSummer holidayFlexible hours$167.1k - $226.1k
...organization, we are building AI systems that process... .... We are looking for a Senior Applied Scientist to lead the development... ...model cost - Developing evaluation frameworks that catch quality regressions before customers... ...at a scale where naive LLM approaches break down...QualitySeniorContract workFlexible hoursAmazon
Bellevue, WA2 days ago- Apple Inc. in Seattle, WA, seeks a Senior Applied Scientist to advance AI evaluation quality systems. You will build scalable ground-truth pipelines, calibration frameworks, and autonomous QA agents to ensure data powering AI/ML systems meets high accuracy, consistency,...QualitySenior
$290.25k
...innovative Machine Learning Scientists to join our AI team, focusing on AI applications (LLM and Computer Vision)... ...architectures. Evaluate the performance of LLM... ...Scientist, +8 years for Senior ML Scientist, or +10 years... ...to deliver high‑quality models enabling multiple...QualitySeniorRemote workNational Society for Black Engineers
Seattle, WA1 day ago$187k
Senior Machine Learning Scientist - Personalization Introduction to the team... ...research into production‑quality systems, influence... ..., model development, evaluation, and iteration, with... ...integrate and operate AI/ML‑enabled solutions... ...semantic IDs, hybrid LLM‑recommender systems,...QualitySenior$187k
...tools to deliver high-quality experiences for travelers... ...of innovation in AI-driven agentic systems.... ...will: Design, build, and evaluate multi-step agentic AI systems... ...ML engineers and scientists, providing technical guidance... ...design patterns, LLM best practices, and experimentation...QualitySeniorWorldwide$167.1k - $226.1k
...We are looking for a Senior Applied Scientist to lead the development... ...manufacturing, test, quality, and operational data... ...models, including LLM-based systems, retrieval... ...approaches, and evaluation criteria. Train, fine... ...learning, and generative AI under real‑world...QualitySeniorWork at officeLocal areaFlexible hoursShift work- Sr. Machine Learning Scientist, Agentic AI & GenAI Systems (Growth Marketing) We create and... .... We are looking for a hands-on Senior Machine Learning Scientist who thrives... ...Agents SDK, etc.) and advanced evaluation platforms leveraging LLM-as-a-Judge, synthetic data...SeniorTemporary work
$173k - $242.5k
Sr. Machine Learning Scientist, Agentic AI & GenAI Systems (Growth Marketing) We build production‑ready multimodal large language... ...AutoGen, OpenAI/Claude Agents SDK, etc.). Build advanced evaluation platforms using LLM‑as‑a‑Judge, synthetic data generation, and agent...SeniorTemporary work$159.75k
...are seeking a skilled and innovative Senior AI Research Scientist to join a new team focusing on agentic... ...visual data. Develop performance and quality metrics for agentic multimodal... ...strategy through model development, evaluation, deployment, and iteration in production...QualitySeniorWork experience placement$134.8k - $245.8k
AI Experience Researcher, Product Evaluation, Vision Products Group Seattle, Washington, United... ...collaborating with ML and data scientists, software engineers,... ...that measure those qualities reliably and at scale. This... ...turn them into prompts for LLM judges or instructions for...QualityRelocation$130k - $180k
...collaborate to advance AI, social media, and investment... ...are seeking an Applied Scientist to advance the... ...Large Language Model (LLM)‑driven systems that power... ...focuses on improving model quality, reliability, and... ...conduct experiments to evaluate model performance, latency...QualityFull timeWork visa- ...leading research accelerator for frontier AI labs and a trusted partner for global... ...accelerating frontier research with high-quality data, advanced training pipelines, plus top... ...Overview What Does a Typical Day Look Like? Evaluate and refine AI-generated code to ensure...QualitySeniorFor contractorsRemote workFlexible hours
Turing
Seattle, WA4 days ago $202.16k - $368.22k
Research Scientist - E-commerce Recommendation(LLM Applications) - Global Frontier Tech Recruitment... ...and purchase high‑quality products at competitive prices... ...applications. Agent Evaluation, Safety & Compliance:... ...knowledge of cutting‑edge AI technologies; publications...QualityTemporary workLocal area- Centific Global Solutions, Inc. is seeking a Research Scientist for LLM Evaluation & Post-Training in Seattle, WA. This full-time role involves leading... ...developing evaluation frameworks and collaborating with AI industry stakeholders. Candidates should have a PhD in a...Full time
- The Opportunity Photoshop ART is seeking a Senior researcher - Machine Learning Systems &... ...services, with a clear mandate: deliver high-quality ML systems at substantially lower cost and... ...in areas such as Artificial Intelligence (AI), ML systems, and computer vision. Strong...QualitySenior
$184.7k - $324.8k
AIML - Senior ML Researcher in Foundation Models, Responsible AI Cupertino, California, United States Machine... ...s values, privacy, and quality standards. Description... ...to train and evaluate foundation models with... ...Qualifications Experience with LLM training and safety...QualitySeniorRelocation$142.7k - $270.95k
...group is looking for research scientists and engineers focused on post... ...of large‑scale generative AI models. The goal is to push the... ...generative AI models. Design and evaluate techniques such as RLHF, DPO/... ...generative models across quality, efficiency, and safety metrics...QualitySeniorTemporary workLocal areaAdobe
Seattle, WA2 days ago$131.5k - $243.3k
...tackle complex data annotation and product evaluation problems. The solutions you develop will... ...which help automate the assessment of quality of data used for training foundational models... ...open source and enterprise-licensed) LLM’s to build grading models and integrate the...QualitySeniorRelocation$214k - $285k
...and unlock the power of AI for thousands of... ...collaboratively and proactively with senior architects, PMs and... ...continuously improve LLM development velocity... ...high level of technical quality. IDEAL REQUIREMENTS &... ...worked well with data scientists, business analysts and...QualitySeniorFlexible hours- Amazon Science in Bellevue, WA, is seeking a Senior Applied Scientist to lead the development of models... ...design, train, and deploy ML models, including LLM-based systems, and work with manufacturing, test, and quality data to drive production decisions. Strong leadership...QualitySenior
- Apple Inc. is seeking a data scientist to research and develop evaluation methods for improving product quality across Siri, Apple Intelligence, and other user-facing features. You will partner with engineering teams to translate methodological developments into practical...QualitySenior
$184.5k
...Senior Applied Economist - Advertising Technology Expedia Group... ...services, and tools to deliver high-quality experiences for travelers,... ...ads are ranked, priced, and evaluated in production. You will work... ...questions Familiarity with AI-driven systems, tools, or workflows...QualitySeniorExpedia Group
Seattle, WA3 days ago$152.82k - $185.18k
Senior Principal Scientist, Drug Product Analytical Science and Technology (ASAT), Cell Therapy Req Number... ..., and optimization. Critically evaluate experimental data across the team, identifying... ...and application within the Quality Control environment; understanding of...QualitySeniorHourly payFull timeSummer workFlexible hours- ...Seattle is seeking an experienced Applied Scientist to lead research and development in Large... ...will possess both a strong background in AI research and practical experience in deploying... ...AI systems, conducting cutting-edge LLM research, and collaborating with cross-functional...Senior
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Scientist, AI Quality & LLM Evaluation. Be the first to apply!
Related searches
- protein scientist Seattle, WA
- lead scientist Seattle, WA
- applied sports scientist Seattle, WA
- deep learning scientist Seattle, WA
- lab scientist Seattle, WA
- molecular biology scientist Seattle, WA
- senior analytical scientist Seattle, WA
- decision scientist Seattle, WA
- image scientist Seattle, WA
- senior research scientist Seattle, WA