Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI/ML Evaluation Specialist, Human Data

$144.6k - $263.8k

Apple Oakbrook

Role Number: 200670744Team: Machine Learning and AILocation: CupertinoPosted: 2026-07-02SummaryAt Apple, we don’t just build products — we build experiences fueled by world-class data. TheHuman-centered AI team within Apple Services Engineering is looking for an ML Evaluation Specialist, Human Data to join our Data Quality and Operations division to spearhead complex, multi-stakeholder operations that specialize in data collection, curation, annotation, and human evaluation efforts across Apple Music, App Store, TV+, Podcasts, and Books.DescriptionIn this role, you will own the operational strategy and continuous improvement of large-scale, multilingual human data programs, from designing onboarding scaffolds that progressively build annotator calibration, to analyzing annotator behavior patterns to identify where automation can offload low-judgment decisions, to enforcing quality frameworks that close the loop between annotator struggle and task redesign. You will identify where human judgment is essential and where it could be better directed, then build the scaffolding, automation, and feedback systems that let annotators focus their cognitive energy where it matters most. Because this work cuts across engineering, data science, research, procurement, and legal, a critical part of the role is serving as the connective tissue between teams who each own a piece of this space, aligning on shared standards, surfacing gaps, and ensuring that insights from the annotation layer inform upstream decisions about task design and tooling. You will bring a point of view on human data best practices and translate it into scalable, human-centered approaches that make generative AI features safer and more reliable.The ideal candidate brings a rare combination of technical depth and program execution skills. You are comfortable designing and deploying sophisticated data pipelines in the morning, and then seamlessly transitioning to present comprehensive quality rectification strategies to stakeholders in the afternoon. You care deeply about data quality and human alignment, have a creative and systematic approach to finding and fixing problems, and find motivation in wide-ranging work whose impact shows up in everyday Apple experiences.Minimum QualificationsBachelor's degree or higher in Cognitive Science, Linguistics, or a related field that includes an experimental or empirical component4+ years of experience defining and leading cross-team human data programs for AI/ML, including annotation operations, quality frameworks, and evaluation strategies, within an NLP/NLU or generative AI environmentProficiency in programming and data languages (Python, R, SQL) to process, analyze, query large datasets, extract insights, automate tasks, and monitor program performanceHands-on experience designing and managing 01 human-in-the-loop data collection, annotation, and evaluation initiatives, including driving and incorporating agentic workflows to improve quality and scalabilityExperience working with diverse data types (e.g., speech, text, multimodal) across multiple languagesExpertise in end-to-end data annotation quality management, including the ability to develop statistical process controls and data quality metricsFamiliarity with privacy-preserving data handling practices and compliance frameworksDemonstrated success optimizing data pipelines and workflows to improve quality, reduce lead time, and scale operationsExperience working cross-functionally with engineering, data science, legal, privacy, and third-party suppliersPreferred QualificationsMaster's degree or higher in Cognitive Science, Linguistics, or a related field that includes an experimental or empirical component2+ years of experience owning data strategy for frontier AI development and evaluation, with experience in human alignment methodologies and agentic GenAI systemsExperience managing external vendor or workforce partners at scaleFamiliarity with AI Safety and Responsible AI principles, including experience applying them to data collection or annotation workflowsStrong organizational skills and execution-oriented mindset; ability to balance attention to detail with big-picture thinking in an environment where program scope and priorities evolve quicklyExcellent written and verbal communication skills; able to translate technical concepts for non-technical stakeholdersPay & BenefitsAt Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $144,600 and $263,800, and your base pay will depend on your skills, qualifications, experience, and location.Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple BenefitsNote: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.EEO StatementApple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicantAccessibilityAt Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.Learn about accessibility in Apple’s workplaceLearn about reasonable accommodations for job applicantsApplication DeadlineApple accepts applications to this posting on an ongoing basis.

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the AI/ML Evaluation Specialist, Human Data in Cupertino, CA vacancy
  • $38 per hour

     ...Overview Welo Data is looking for experienced, detail...  ...with both people and AI systems — auditing outputs...  ..., and edge cases in both human and model outputs Participate...  ...~ Experience in AI/ML data workflows or human‑in‑the‑loop evaluation environments ~ Prior experience... 
    Data
    Full time
    Contract work
    Remote work
    Visa sponsorship

    Welo Global

    Sunnyvale, CA
    2 days ago
  •  ...generation computing experiences—from AI and data centers, to PCs, gaming and...  ...comes from bold ideas, human ingenuity and a shared...  ...THE ROLE We are hiring AI / ML Platform Engineers to build the...  ...distributed inference, batch evaluation, and large-scale agent rollout... 
    Data
    Full time

    AMD

    Santa Clara, CA
    12 days ago
  • $185k

     ...senior leader to own the Data Trust and Governance on...  ...data platform, and expand AI-driven fully self-...  ...aggregated and explainable for human users, while also being...  ...and trust for AI/ML and agentic AI, including...  ...contextual, grounded, and evaluation data. ~ Experience with... 
    Data
    Full time
    Contract work
    Work at office
    Remote work
    Shift work

    Uber

    Sunnyvale, CA
    13 days ago
  • $207k - $275k

     ...The Essential Cloud for AI™. Built for pioneers by...  ...-maker. As a Solution Specialist for AI Runtime Services...  ...in distributed systems, ML infrastructure, or...  ...-stakeholder technical evaluations without losing momentum...  ...day in our office and data center locations ~ A... 
    Data
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    Sunnyvale, CA
    1 day ago
  • $150.4k - $277.6k

     ...looking for an exceptional ML Engineer to help us...  ...generation of scalable evaluation infrastructure and lead...  ...cutting-edge tools, synthetic data pipelines, and automated...  ...are passionate about AI safety, robust software...  ...accessibility is a fundamental human right. You’ll find that... 
    Data
    Relocation

    Apple

    Cupertino, CA
    3 days ago
  • $184k - $287.5k

     ...can do. Now, generative AI is opening new possibilities...  ...in machine learning for Evaluations, you’ll help establish...  .../ AI Researcher / Data Scientist to join our AI...  ...coverage, incorporating human review to validate automated...  ...statistically sound comparisons for ML systems, including... 
    Data
    Full time

    Nvidia

    Santa Clara, CA
    14 hours ago
  • $184.7k - $324.8k

     ...you get excited by building AI applications to enhance the evaluation of various Apple AI...  ...in leveraging advanced AI/ML techniques to enhance both...  ...product, annotation, evaluation data scientists, autograder...  ...generation.Familiarity with human annotation operations, sampling... 
    Data
    Relocation

    Apple

    Cupertino, CA
    3 days ago
  • $184.7k - $277.6k

     ...Team: Machine Learning and AI, Sales and Business...  ...seeking a visionary AI Evaluations Engineer to own the end-...  ...years of experience in data and AI-related fields such...  ..., software development, ML engineering, data science...  ...accessibility is a fundamental human right. You’ll find that... 
    Data
    Relocation

    Apple

    Cupertino, CA
    3 days ago
  • $184.7k - $324.8k

     ...the next revolution in human-computer interaction and...  ...interaction!Apple's Siri AI Quality team is looking...  ...and skilled AI Software Evaluation Engineer to join our mission...  ...agentic engineering, data analytics and AI evaluation...  ..., data engineering or ML engineering.2+ years of... 
    Data
    Worldwide
    Relocation
    Shift work

    Apple

    Cupertino, CA
    14 hours ago
  • $237.6k - $356.4k

     ...leader with deep expertise in data platforms, AI data engineering, and modern...  ...close partnership with AI/ML, product, and business teams...  ...curation, validation, quality evaluation, metadata management,...  ...accessibility is a fundamental human right. You’ll find that idea... 
    Data
    Relocation

    Apple

    Cupertino, CA
    3 days ago
  • $150.4k - $277.6k

     ...Camera & Photos Tools & AI team is a tight-knit engineering...  ...Quality teams measure, evaluate, and improve the imaging...  ...backend services and data infrastructure end to...  ...for our AI/ML models. The ideal candidate...  ...accessibility is a fundamental human right. You’ll find that... 
    Data
    Relocation
    Shift work

    Apple

    Cupertino, CA
    3 days ago
  • $184.7k - $324.8k

     ...experience to determine how agentic AI, multimodal vision, robotics,...  ...and iterate on ideas, use data to drive decisions, and work...  ...device automation, and human-in-the-loop workflows that scale...  ...hands-on experience building and evaluating modern AI/ML and agentic technologies to... 
    Data
    Work experience placement
    Relocation
    Shift work

    Apple

    Cupertino, CA
    3 days ago
  • $267.8k - $401.7k

     ...Services Engineering (ASE) powers AI and machine learning...  ...lead AI Safety Engineering & Evaluation within the Human-Centered AI organization. This...  ...of complex software or AI/ML solutions.Practical expertise...  ...model assessors, synthetic data generation, red teaming, or... 
    Data
    Relocation

    Apple

    Cupertino, CA
    3 days ago
  • $124.92k - $171.77k

     ...including high-growth ones in AI datacenters, automated...  ...models with SiTime data, knowledge, and core systems...  ...prototyping through evaluation, deployment, monitoring,...  ...reliability, and cost; add human review where appropriate...  ...3+ years building AI/ML or generative AI solutions... 
    Data

    SiTime

    Santa Clara, CA
    4 days ago
  • $110k - $170k

     ...turn large volumes of operational data into reliable, auditable workflows - and applies LLM-based evaluation only where output quality can...  ..., idempotency, evaluation, human review - to every workflow you...  ...of late or incomplete data AI Workflows & Evaluation Design... 
    Data
    Full time
    Contract work
    Internship
    Worldwide
    Home office

    Fortinet

    Sunnyvale, CA
    1 day ago
  • $150.4k - $277.6k

     ...across global factory sites.As an AI Data Platform Engineer with the MSI...  ...pipelines, observability, and evaluation metrics to ensure trusted AI...  ...datasets.Collaborate with AI/ML engineers, software engineers,...  ...is a fundamental human right. You’ll find that idea reflected... 
    Data
    Relocation

    Apple

    Cupertino, CA
    3 days ago
  • $149.2k - $249k

     ...critical in building the data infrastructure to...  ...initiatives.DescriptionAs an AI Data & Knowledge Engineer...  ...Strong understanding of LLM evaluation and AI quality tooling,...  ....Familiarity with AI/ML model development lifecycle...  ...is a fundamental human right. You’ll find that... 
    Data
    Relocation

    Apple

    Cupertino, CA
    3 days ago
  •  ...with citations to source data. Design and implement...  ...and cost. Implement evaluation and continuous monitoring...  ...harnesses, golden sets, human-in-the-loop review,...  ...vector stores (e.g., Azure AI Search) Prompt design:...  ...years of experience in AI/ML engineering, data... 
    Data

    Cloud Analytics Technologies LLC

    Palo Alto, CA
    22 days ago
  •  ...Senior WW Partner GTM Specialist For Agentic Ai The AWS Data and AI Strategic Partner GTM team supports the world's most innovative partners - from breakthrough...  ...related technologies, including, but not limited to, AI/ML, GenAI, Analytics, Database, and/or Storage experience... 
    Data
    Worldwide

    A W S

    Mountain View, CA
    2 days ago
  •  ...peoples lives. We are seeking a Senior Human-Centered AI Researcher to help shape the next...  ...closely with engineering design marketing data science and AI/ML modeling teams you will define...  ...understanding user needs and expectations to evaluating early prototypes and assessing... 
    Data
    Full time

    Apple

    Cupertino, CA
    3 days ago
  • $153.6k - $207.8k

     ...expertise to help customers evaluate, design, and adopt AWS as their...  ...platform for Generative AI and ML workloads? Do you enjoy building...  ...solutions? Join the GenAI/ML Specialists Solution Architect team as a...  ...architects, product leaders, data scientists, and executives at... 
    Data
    Local area
    Worldwide
    Flexible hours

    AmazonWebServices

    Mountain View, CA
    14 hours ago
  • $175k - $275k

     ...seeking an experienced Senior AI Engineering to design,...  .../function calling, and human-in-the-loop controls....  ...Flask, Azure OpenAI, Azure ML, Databricks, ADLS, and...  ...CI/CD, testing, model evaluation, observability, performance...  ...Partner with business, data, cybersecurity, risk,... 
    Data

    Unavailable

    Palo Alto, CA
    3 days ago
  • $53 - $70 per hour

     ...work on cutting-edge Agentic AI research while tackling real-world...  ..., vision, and structured data Human-agent collaboration and interactive AI systems Agent evaluation, benchmarking, reliability, and...  ...and experience with modern AI/ML framework Familiarity with... 
    Data
    Hourly pay
    Work experience placement
    Internship
    Local area

    Bosch

    Sunnyvale, CA
    4 days ago
  •  ...accelerates our organizations AI ambitions by leading the...  ...scalable and innovative human-centered AI solutions...  ...is backed by rigorous data-driven evidence....  ...retrieval and failure and run evaluations: work with domain experts...  ...in Computer Science AI/ML Data Science or a related... 
    Data
    Full time
    Internship

    Apple

    Cupertino, CA
    9 days ago
  • $201.3k - $352.3k

     ...work. Today, ServiceNow is the AI control tower for business...  ...platform brings together any AI, any data, and any workflow— helping 85%...  ...& GenAI Benchmarking and Evaluations to establish and lead AI evaluation...  ...leading high-performing AI/ML teams. ~ Experience in... 
    Data
    Full time
    Work experience placement
    Work at office
    Immediate start
    Remote work
    Flexible hours
    Shift work

    ServiceNow

    Santa Clara, CA
    12 days ago
  • $136k - $245k

     ...the world’s most advanced AI-native platform. We work...  ...from problems with less human intervention: dependency...  ..., not an AI research or ML role. You'll write...  ...quickly. Build the APIs, data sources, and integrations...  ...code. Constantly re-evaluate our platform to improve... 
    Data
    Permanent employment
    Full time
    Work experience placement
    Work at office
    Local area
    3 days per week
    1 day per week
    Sunnyvale, CA
    9 hours ago
  • $45 - $65 per hour

     ...improve the quality of human life. We’re developing new...  ...the state of the art in AI, robotics, driving, and...  ...are an integrated team of ML researchers and social...  ...AI/ML, computer science, data science, statistics, or...  ...with one or more of: LLM evaluation and benchmarking;... 
    Data
    Internship
    Work at office
    Local area
    Shift work

    Toyota Research Institute

    Los Altos, CA
    1 day ago
  • $45 - $65 per hour

     ...improve the quality of human life. We’re developing new...  ...the state of the art in AI, robotics, driving, and...  ...and develop methods for evaluating how well these models capture...  ...in related fields - AI/ML, computer science,...  ...information about how your data is processed, please... 
    Data
    Summer work
    Internship
    Work at office
    Local area
    Shift work

    Toyota Research Institute

    Los Altos, CA
    13 days ago
  •  ...deployment of embodied AI models , including Vision...  ..., large-scale robot data, and model deployment. This...  ...datasets and egocentric human demonstration data ....  ...signals Develop and evaluate multimodal fusion architectures...  ...with relevant ML/robotics tooling. ~ Understanding... 
    Data

    Maxinsights

    Santa Clara, CA
    12 days ago
  • $34 per hour

     ...Description Job Description Overview Welo Data is looking for sharp, curious, and...  ...be working directly with cutting-edge AI systems — evaluating outputs, identifying gaps, and helping...  ...data quality, model evaluation, and human judgment , where your ability to think... 
    Data
    Full time
    Remote work
    Visa sponsorship

    Welo Data

    Sunnyvale, CA
    24 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI/ML Evaluation Specialist, Human Data. Be the first to apply!