ML Engineer, Proactive - Agentic Systems Evaluation

ML Engineer, Proactive - Agentic Systems Evaluation

Are you passionate about working on the next generation of personalized intelligence systems? In this role, you will be developing and deploying robust evaluation frameworks across the data lifecycle -- from data collection and processing, to analytic dashboards for reporting. You will be part of the larger Proactive Intelligence team, which builds features that anticipate customer's needs and create personalized experiences by adapting to user behaviors with machine learning running locally on-device or in PCC. Join our cross functional team of specialists dedicated to the evaluation of agentic systems.

Description

We are looking for a high-impact ML Evaluation Engineer to help architect rigorous evaluations systems for autonomous agents. With the rise of generative AI, the ability to quantify the reliability and quality of these systems is more critical than ever. You will design and deploy qualitative and quantitative metrics to measure the quality, reasoning, and tool-use accuracy of agentic systems. You will be working with very sensitive data, so leveraging existing and developing new privacy enhancing technologies -- such as differential privacy, PII redaction, and data minimization -- will be crucial. The team you will be joining is focused on advancing scalable automated processes for evaluation. To succeed, you will need a deep understanding of system-level software operations to deliver next-generation capabilities. Join the Proactive Intelligence team to build the evaluation platforms for the future of intelligent, personalized experiences.

Responsibilities

• Design and implement evaluation frameworks to measure quality, reasoning, and tool-use accuracy of agentic systems
• Develop MCP servers and API orchestration layers to enable reliable tool-use for agentic systems.
• Orchestrate end-to-end ML workflows by integrating heterogeneous internal systems — spanning data services, compute infrastructure, model deployment, and results visualization — into cohesive, production-ready pipelines
• Create and manage analytic dashboards to surface evaluation insights to key stakeholders.
• Collaborate cross-functionally with various teams across ML and SWE teams.

Minimum Qualifications

• MS or PhD in Computer Science, Machine Learning, Statistics, or equivalent practical experience in a quantitative field.
• 3+ years of industry experience in ML Engineering or Applied Science.
• Strong software engineering fundamentals (Python is a must) with experience building scalable, automated data or evaluation pipelines.

Preferred Qualifications

• Demonstrated experience applying Differential Privacy, Federated Learning, or advanced PII redaction techniques to large-scale datasets.
• Hands-on experience building or testing LLM-based systems, including a deep understanding of chain-of-thought reasoning, prompt engineering, and agentic planning.
• Proficiency in building or evaluating systems that integrate with external tools/APIs.
• Experience with specialized agent evaluation frameworks and analyzing execution traces to identify failure modes in multi-turn interactions.
• Experience with compiled languages (e.g., Swift) and a curiosity about how ML interacts with OS-level software operations.
• A track record of developing custom metrics (e.g., "LLM-as-a-Judge") or publishing research on model reliability, safety, or algorithmic bias.

Pay & Benefits

At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $126,800 and $220,900, and your base pay will depend on your skills, qualifications, experience, and location. Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses — including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program. Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant At Apple, we believe accessibility is a fundamental human right. You'll find that idea reflected in everything here — in our culture, our benefits and our digital tools. By welcoming as many perspectives as possible, we help you build a career where you feel like you belong. Learn about accessibility in Apple's workplace Learn about reasonable accommodations for job applicants Apple accepts applications to this posting on an ongoing basis.