AI/ML Evaluation Specialist, Human Data

Role Number: 200670744Team: Machine Learning and AILocation: CupertinoPosted: 2026-07-02SummaryAt Apple, we don’t just build products — we build experiences fueled by world-class data. TheHuman-centered AI team within Apple Services Engineering is looking for an ML Evaluation Specialist, Human Data to join our Data Quality and Operations division to spearhead complex, multi-stakeholder operations that specialize in data collection, curation, annotation, and human evaluation efforts across Apple Music, App Store, TV+, Podcasts, and Books.DescriptionIn this role, you will own the operational strategy and continuous improvement of large-scale, multilingual human data programs, from designing onboarding scaffolds that progressively build annotator calibration, to analyzing annotator behavior patterns to identify where automation can offload low-judgment decisions, to enforcing quality frameworks that close the loop between annotator struggle and task redesign. You will identify where human judgment is essential and where it could be better directed, then build the scaffolding, automation, and feedback systems that let annotators focus their cognitive energy where it matters most. Because this work cuts across engineering, data science, research, procurement, and legal, a critical part of the role is serving as the connective tissue between teams who each own a piece of this space, aligning on shared standards, surfacing gaps, and ensuring that insights from the annotation layer inform upstream decisions about task design and tooling. You will bring a point of view on human data best practices and translate it into scalable, human-centered approaches that make generative AI features safer and more reliable.The ideal candidate brings a rare combination of technical depth and program execution skills. You are comfortable designing and deploying sophisticated data pipelines in the morning, and then seamlessly transitioning to present comprehensive quality rectification strategies to stakeholders in the afternoon. You care deeply about data quality and human alignment, have a creative and systematic approach to finding and fixing problems, and find motivation in wide-ranging work whose impact shows up in everyday Apple experiences.Minimum QualificationsBachelor's degree or higher in Cognitive Science, Linguistics, or a related field that includes an experimental or empirical component4+ years of experience defining and leading cross-team human data programs for AI/ML, including annotation operations, quality frameworks, and evaluation strategies, within an NLP/NLU or generative AI environmentProficiency in programming and data languages (Python, R, SQL) to process, analyze, query large datasets, extract insights, automate tasks, and monitor program performanceHands-on experience designing and managing 01 human-in-the-loop data collection, annotation, and evaluation initiatives, including driving and incorporating agentic workflows to improve quality and scalabilityExperience working with diverse data types (e.g., speech, text, multimodal) across multiple languagesExpertise in end-to-end data annotation quality management, including the ability to develop statistical process controls and data quality metricsFamiliarity with privacy-preserving data handling practices and compliance frameworksDemonstrated success optimizing data pipelines and workflows to improve quality, reduce lead time, and scale operationsExperience working cross-functionally with engineering, data science, legal, privacy, and third-party suppliersPreferred QualificationsMaster's degree or higher in Cognitive Science, Linguistics, or a related field that includes an experimental or empirical component2+ years of experience owning data strategy for frontier AI development and evaluation, with experience in human alignment methodologies and agentic GenAI systemsExperience managing external vendor or workforce partners at scaleFamiliarity with AI Safety and Responsible AI principles, including experience applying them to data collection or annotation workflowsStrong organizational skills and execution-oriented mindset; ability to balance attention to detail with big-picture thinking in an environment where program scope and priorities evolve quicklyExcellent written and verbal communication skills; able to translate technical concepts for non-technical stakeholdersPay & BenefitsAt Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $144,600 and $263,800, and your base pay will depend on your skills, qualifications, experience, and location.Apple employees also have the opportunity to become an Apple shareholder through participation in Apple’s discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan. You’ll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple BenefitsNote: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.EEO StatementApple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicantAccessibilityAt Apple, we believe accessibility is a fundamental human right. You’ll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong.Learn about accessibility in Apple’s workplaceLearn about reasonable accommodations for job applicantsApplication DeadlineApple accepts applications to this posting on an ongoing basis.