Sr Machine Learning Engineer, Tech Lead Autograder Systems, Evaluation

Sr Machine Learning Engineer, Tech Lead — Autograder Systems, Evaluation

We are looking for a Senior MLE Tech Lead to join a centralized evaluation organization and define the next generation of autograder quality across 20+ of Apple's most visible generative AI features. You will own the end-to-end technical vision for how we evaluate model outputs at scale — pioneering state-of-the-art methods, raising the technical bar, and leading a team of talented MLEs to build a robust autograder training and hillclimbing system from the ground up. This is a high-impact, hands-on leadership role at the intersection of model evaluation, data quality, and ML systems engineering. You will work closely with model developers, data teams, and product partners to ensure our autograders are fast, accurate, and continuously improving — directly shaping the quality of AI experiences used by hundreds of millions of people.

In this role you will focus on:

• Technical Leadership
• Define and drive the technical roadmap for autograder quality — researching and introducing novel methods such as reward modeling, LLM-as-judge, preference learning, and calibration techniques to measurably improve evaluation accuracy.
• Architect and lead the build-out of a scalable autograder training pipeline encompassing data curation, model fine-tuning, evaluation harnesses, and versioning.
• Design and own the hillclimbing system that iteratively improves autograder performance through systematic prompt and model optimization loops.
• Establish quality benchmarks, confidence metrics, and failure analysis frameworks that enable the team to track, trust, and act on autograder outputs.
• People & Collaboration
• Mentor and technically guide a team of MLEs through design reviews, modeling standards, and hands-on problem-solving — fostering a culture of rigor and continuous learning.
• Partner with data annotation teams to define labeling guidelines that feed autograder training.
• Collaborate with feature engineers to align autograder signals with broader training and product objectives.
• Translate complex technical trade-offs into clear narratives for engineering, product, and leadership audiences.

Minimum Qualifications

• Master's or PhD in Computer Science, Machine Learning, Artificial Intelligence, or a related field.
• 5+ years of industry experience in machine learning, with a strong focus on LLM or VLM systems.
• Deep expertise in prompt-tuning and fine-tuning techniques (SFT, RLHF, DPO, or equivalent), with proven experience of model calibration and uncertainty estimation.
• Familiarity with data flywheel design — leveraging model outputs to continuously improve future training data.
• Proficiency in Python and ML frameworks (PyTorch preferred).

Preferred Qualifications

• Strong ML systems instincts — you care deeply about data quality, reproducibility, latency, and scale.
• Background in human-in-the-loop annotation pipelines and inter-annotator agreement analysis.
• Prior experience on an evaluation infrastructure or model quality team.

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $181,100 and $318,400, and your base pay will depend on your skills, qualifications, experience, and location. Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program. Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant At Apple, we believe accessibility is a fundamental human right. You'll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong. Learn about accessibility in Apple's workplace Learn about reasonable accommodations for job applicants Apple accepts applications to this posting on an ongoing basis.