Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AIML Sr Machine Learning Platform Engineer, Evaluation

Full-time

Apple Oakbrook

We are seeking a highly skilled and experienced machine learning engineer to join AIML Evaluation to build the systems that evaluate and refine Apples foundation models and agents. nAs a key member of the team you will help design and develop benchmarks evaluators simulation environments and prompt and context optimization pipelines that drive quality improvements across Apples AI experiences. nYou will collaborate with product teams and the foundation model team to close the loop between observation and improvement contributing datasets environments and reward signals that drive model and agent quality.

Most model evaluation stops at a scorecard. The failing traces get thrown away even though theyre the most relevant training signal we have. Our team puts them to work. We evaluate across offline benchmarks device-in-the-loop simulation and on-device observation in production and we turn what we find into persona-grounded datasets and reward signals for foundation model this role you will build the systems that connect failure analysis to measurable model and agent improvements across a broad range of AI products at Apple. The path from finding a problem to fixing it is short: the loop you build finds the failure produces the data and checks the result. The work reaches millions of users.

Designing and building evaluation infrastructure for agents and foundation technical direction for the evaluation system and bringing partner teams LLM judges reward models calibrated against human prompt and context optimization and integrating simulation environments for agent evaluation and trajectory-based data with product and research teams to identify prioritize and address quality shared foundations such as run lineage a run registry and MCP interfaces for client teams agents.

Strong background in machine learning and distributed proven track record of solving ambiguous problems end to end: defining the problem and the metric building the solution and showing measured leading technical work across multiple codebases teams and building and maintaining ML infrastructure for evaluation training or deployment.n 6 years of professional experience as a software engineer preferably in machine learning or a related field. Bachelors or Masters degree in Computer Science or a related field or equivalent practical experience.

Experience with LLM evaluation LLM-as-judge or reward with prompt optimization agent harness development or post-training (SFT DPO RLHF).nProficiency in Python and ML frameworks such as PyTorch or with agentic systems simulation environments or trajectory-based data with on-device or privacy-preserving of how training data is used in post-training and how data choices affect model communication skills.

Required Experience:

Senior IC

Vacancy posted 8 days ago
Similar jobs that could be interesting for youBased on the AIML Sr Machine Learning Platform Engineer, Evaluation in Washington DC vacancy
  • We are seeking a highly skilled and experienced machine learning engineer to join AIML Evaluation to build the systems that evaluate and refine Apple's foundation models and agents. As a key member of the team, you will help design and develop benchmarks, evaluators, simulation... 
    Senior

    Apple

    Washington DC
    9 days ago
  • $272.7k - $356.03k

     ...Responsibilities Your future team: Search Platform The Search Platform team is...  ...in information retrieval and machine learning into production systems through rigorous evaluation, experimentation, and...  ...systems, mentor senior engineers, and align technical and product... 
    Senior
    Full time
    Work at office
    Local area

    Atlassian

    Washington DC
    11 days ago
  •  ...generation of our modeling and prediction engine. The applicant must have a strong focus...  ...~ Construct and fit statistical, machine learning, or vision models ~ Implement advanced...  ...Design experimentation strategies for evaluating our models ~ Collaborate with developers... 
    Senior
    Full time
    Work experience placement
    Work at office

    Strella

    Washington DC
    3 days ago
  • $95k - $141k

     ...analysis with our cutting-edge technology platform, Narrative Analytics®. This powerful...  ...Protagonist is looking for a Senior Machine Learning Engineer who builds production-ready systems...  ...-language content  Establish ML evaluation frameworks regression testing, hallucination... 
    Senior

    Protagonist

    Washington DC
    23 days ago
  • $229.9k - $262.4k

    Sr. Manager, AI Engineer (Gen AI Platform Services: Agentic AI, Guardrails, Evaluation) At Capital One, we are creating responsible and reliable AI systems, changing banking...  ...One has been an industry leader in using machine learning to create real-time, personalized... 
    Senior
    Full time
    Part time
    Local area

    Capital One

    McLean, VA
    1 day ago
  •  ...As a Principal Recommendation System/Platform Engineer, you will lead the upgrade of our core...  ...engineering feedback loop from offline evaluation and online experimentation to production...  ...with at least one mainstream deep learning framework, such as PyTorch or TensorFlow... 
    Full time

    ByLabs

    Washington DC
    9 hours ago
  • $90 - $102 per hour

     ...Senior Machine Learning Engineer (ML Modeling, NLP, Python)18 Months Contract Onsite 4 days/week in Seattle WA or Santa Monica CA Summary...  ...metadata and related data to develop features, train and evaluate classifiers, and build scalable workflows that generate... 
    Senior
    Full time
    Contract work
    For contractors

    Milestone Technologies, Inc.

    Washington DC
    2 days ago
  • $185k - $235k

     ...in their zip code. Three platforms with one vision: powering...  ...and speak up. Join us. Learn more at: NewsBreak, Nearby...  ...for an experienced Senior Machine Learning Engineer to design and build the data...  ..., training, offline/online evaluation, deployment, monitoring, and... 
    Senior
    Full time
    Local area

    NewsBreak

    Washington DC
    3 days ago
  • $168.1k - $227.4k

     ...Prediction team builds the machine learning systems that decide which ads...  ...across every Amazon platform and surface worldwide. Youll...  ...take models from training and evaluation through deployment and serving...  ...independently - Mentor engineers across the team and be the... 
    Senior
    Full time
    Internship
    Local area
    Immediate start
    Worldwide
    Flexible hours

    Amazon

    Washington DC
    7 days ago
  • $150k - $172k

     ...Title: Sr. Machine Learning Engineer, Autonomy Company: Heven AeroTech Location: Bingen, WA  *On...  ...responsibilities and works closely with Platform Engineers, Flight Test, and aircraft...  ...-time and near-real-time execution. Evaluate and optimize latency, throughput,... 
    Senior
    Permanent employment
    Temporary work
    Flexible hours

    Heven AeroTech

    Washington DC
    4 hours ago
  • $150.75k - $241.2k

     ...matter. Your Impact As an Senior Machine Learning Engineer II focused on Sensor Fusion & Tracking...  ...sensing modalities. Develop and evaluate algorithms such as Kalman filters (EKF...  ...message, WhatsApp, or instant messaging platforms without a formal interview process.... 
    Senior
    Work experience placement
    Work at office
    Remote work
    Worldwide

    Axon

    Washington DC
    4 hours ago
  • $170k - $240k

     ...company.  Our extensive learning programs and...  ...member of the Product and Engineering team at PitchBook, you...  ...with us!  As a Senior Machine Learning Engineer (MLE...  ...features on the PitchBook Platform. You will be deeply...  ..., monitoring, evaluation, and compliance. Help... 
    Senior
    Work at office
    Remote work
    Visa sponsorship

    PitchBook Data

    Washington DC
    4 hours ago
  • $128.8k - $214.5k

     ...Test and Evaluation AI Expert Unlock the secrets of intelligence...  ...Design and develop machine learning algorithms, deep learning models...  ...data preprocessing, feature engineering, model training, validation...  ...Knowledge of cloud computing platforms (e.g., AWS, Azure, Google... 
    Hourly pay
    Contract work
    Temporary work
    Work experience placement
    Local area
    Remote work

    MANTECH

    Washington DC
    15 hours ago
  • $196k - $245k

    The goal of a Senior Machine Learning Engineer at Scale is to own how we apply generative AI, agentic...  ...over maps and spatial data, and the evaluation tooling that benchmarks and refines agent...  ...working with cloud platforms (eg. AWS or GCP) and deploying machine... 
    Senior
    Full time

    Scale AI

    Washington DC
    9 hours ago
  •  ...interactive mobile advertising platform. Focused on the APAC region...  ...systems using advanced machine learning, deep learning, and optimization...  ...for Machine Learning Engineers who are passionate about applying...  ...making problems. Design and evaluate experiments to improve model... 

    Mintegral

    Washington DC
    4 days ago
  •  ...Mission Starts Here TheIncLab engineers and delivers intelligent digital applications and platforms that revolutionize how our...  ...We are looking for a Senior Machine Learning Engineer to that will focus...  ..., designing, training, and evaluating machine learning models to... 
    Senior
    Flexible hours

    TheIncLab

    McLean, VA
    29 days ago
  •  ...About the Role As a Principal Recommendation Platform Engineer, you will help upgrade our core recommendation architecture and feature...  ...system into an industrial-grade, large-scale, real-time machine learning recommendation platform. The role spans key business scenarios... 
    Full time

    ByLabs

    Washington DC
    9 hours ago
  • $153.9k - $300.96k

     ...product understanding, and machine learning solutions that impact commerce...  ...with product managers, engineers, and cross-functional teams...  ...experimentation, model iteration, evaluation, and deployment....  ...and unique perspectives. Our platform connects people from across... 
    Full time
    Temporary work
    Work experience placement
    Internship
    Local area

    TikTok

    Washington DC
    11 days ago
  • $146.7k - $191.53k

     ...where we have a legal entity. Learn how we work at [go.atlassian....  ...Your future team Atlassian’s machine learning teams tackle product...  ...partners across Product, Engineering, Growth, GTM, Finance, and Sales...  ...Responsibilities Build and evaluate machine learning models and... 
    Full time
    Internship
    Local area

    Atlassian

    Washington DC
    11 days ago
  •  ...AI-powered drone software platform, and HX-2, our autonomous drone...  ...annotation, training, evaluation, and deployment to edge platforms...  ...Collaborating with systems engineers to integrate models into...  ...of experience in applied machine learning or computer vision   ~ Have... 
    Local area
    Remote work
    Flexible hours

    Helsing

    Washington DC
    3 days ago
  • $229.9k - $262.4k

    Senior Manager, Machine Learning Engineer Do you love building and pioneering in the AI and technology...  ...Build and scale massive multi-tenant platforms that enable running large footprint...  ..., regularization), and how to evaluate model accuracy and diagnose and address... 
    Senior
    Full time
    Part time
    Local area

    Capital One

    McLean, VA
    1 day ago
  •  ...Payment Model Implementation and Evaluation plays a pivotal role in the...  ...and technology teams, the Sr. Researcher conducts analytics...  ...capabilities of the Commence platform, driving greater efficiency...  ...economists, data scientists, and engineers to conduct research, create... 
    Senior
    Remote work

    Commence

    Washington DC
    7 days ago
  • Eagle Technologies, Inc. is seeking a Principal Investigator to direct evaluations of priority behavioral health programs for federal clients, including SAMHSA, CMS, and NIH. The role covers evaluation design, instrument development, data analysis, and reporting, with emphasis... 

    Eagle Technologies, Inc.

    Arlington, VA
    4 days ago
  • $222k - $300k

     ...and AI infrastructure platform so our customers can use...  ...business. Founded by engineers — and customer...  ...across millions of virtual machines. And we're only getting...  ...and over 300 million evaluations per second at ~10μs p9...  ...and Unity Catalog. To learn more, follow Databricks... 
    Senior
    For contractors
    Local area
    Worldwide

    Databricks

    Washington DC
    3 days ago
  • $168.1k - $227.4k

    Prime Video Personalization and Discovery (PVPD) is seeking a Senior Machine Learning Engineer to lead Personalization and Discovery customer experiences. This role will work across team boundaries within PVPD and with external partners to drive the success of AI-powered... 
    Senior
    Full time
    Internship
    Flexible hours

    Amazon

    Washington DC
    3 days ago
  • $160k - $250k

     ...Deploy is an intelligence infrastructure platform that gives developers and enterprises...  ...operates an optimized model inference engine that keeps costs and latency low while...  ...Seattle, New York, and Delhi.   Senior Machine Learning Engineer   In order to execute our... 
    Senior
    Full time

    Hive

    Washington DC
    12 days ago
  •  ...while staying connected and building community through regular in-person events. Learn more about our flexible approach to where we work. About the Role: As a Machine Learning Engineer, you will have the opportunity to collaborate closely with senior engineers and... 
    Permanent employment
    Full time
    Work at office
    Remote work
    Work from home
    Flexible hours

    Instacart

    Largo, MD
    5 days ago
  • $107.66k - $161.7k

     ...: a global knowledge sharing platform with over 400M monthly unique...  ...providing a unique platform to learn and connect with others....  ...Team and Role: Our small engineering team works on challenging problems...  ...company. At Poe, we use Machine Learning in various parts of... 
    Remote job
    Full time
    Work experience placement
    Internship

    Quora

    Adelphi, MD
    5 days ago
  • $229.9k - $262.4k

    Machine Learning Engineer 5 Do you love building and pioneering in the AI and technology space? Do you...  ...Build and scale massive multi-tenant platforms that enable running large footprint...  ...hyperparameters, regularization), and how to evaluate model accuracy and diagnose and... 
    Full time
    Part time
    Local area

    Capital One

    McLean, VA
    8 hours ago
  • $202k

     ...Overseeing full project lifecycles spanning initial scoping, offline evaluation, experimental testing, production deployment, and post-launch...  ...+ Bachelor’s degree or equivalent in Computer Science, Engineering, Mathematics or related field, with 4+ years of full-time... 
    Senior
    Full time
    Work at office
    Remote work

    Uber

    Washington DC
    8 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AIML Sr Machine Learning Platform Engineer, Evaluation. Be the first to apply!