Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

ML Engineer - Model Evaluation Expert

$60 - $90 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Machine Learning Engineer — Model Evaluation & Experimentation
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Design tasks by transforming real ML research ideas into well-defined, multi-step tasks.
  • Run experiments by implementing changes, executing training experiments, and analyzing results to define correct solutions.
  • Explore reinforcement learning concepts such as reward functions and training behavior in task development.
  • Evaluate frontier models' performance on tasks and identify areas of improvement.
  • Collaborate with researchers to ensure tasks are consistent, rigorous, and fair.

Qualifications

Must-Have

  • MSc or PhD in machine learning , computer science , or a related STEM field.
  • 1+ years in a research or research-engineering role.
  • Experience in training and evaluating ML models and conducting end-to-end experiments.
  • Proficiency in Python and Git .
  • Strong understanding of large language models and their evaluation.

Preferred

  • Basic knowledge of reinforcement learning .
  • Experience in AI training, model evaluation, or benchmark/task authoring.

Compensation & Legal

  • Hourly contractor
  • Paid weekly

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 10 days ago
Similar jobs that could be interesting for youBased on the ML Engineer - Model Evaluation Expert in San Francisco, CA vacancy
  •  ...an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance...  ...and safety teams to refine evaluation rubrics and improve model alignment. Must have 5+ years in AI Safety or related fields and... 
    Suggested
    Remote job
    Contract work
    Flexible hours

    Mercor

    San Francisco, CA
    3 days ago
  •  ...construction veterans and world-class engineers to solve physical-world...  ...Machine Learning Engineer: Evaluation Bedrock is bringing...  ...experience evaluating complex ML systems deployed in the real...  ...evaluation use cases. Leverage models to source rich annotations for... 
    Suggested
    Full time
    For contractors

    bedrock-robotics

    San Francisco, CA
    4 days ago
  • $180k - $220k

     ...careers alongside a team of experts across energy,...  ...Crusoe is hiring an AI Engineer for our Developer Relations...  ...workflows. Test features and models before launch, make...  ...a software engineer, ML engineer, applied AI engineer...  ...retrieval, memory, and evaluation, using one or more... 
    Suggested
    Temporary work

    Crusoe

    San Francisco, CA
    15 days ago
  • $184k - $272k

     ...and the operators and workflow engines that keep long-running...  ...what the ecosystem is shipping, evaluate honestly, and make the case with...  ...dashboards researchers trust, a real model registry, and lineage from...  ...Build observability across the ML stack — utilization,... 
    Suggested
    Full time
    Live in
    Work at office
    Local area
    Remote work
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    24 days ago
  • $184k - $272k

     ...and efficiency. Comprehensively profile model runtime and memory to pinpoint performance bottlenecks. Identify and evaluate emerging technologies that can be adopted...  ...techniques. Work with researchers and ML engineers on best-practices for optimal resource usage... 
    Suggested
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    12 days ago
  •  ...Job title: ML Engineer / Researcher, Large Language Models WORK AUTHORIZATION Must be authorized to work in the U.S.; we are unable to sponsor visas...  .... You will design experiments, build datasets and evaluation systems, and fine-tune and post-train models. You... 
    Full time
    Work at office
    Visa sponsorship

    knowtex

    San Francisco, CA
    4 days ago
  • $90 - $140 per hour

     ...advanced Large Language Models. 1\. Overview Join a...  ...talented Machine Learning Engineers and Scientific Computing experts with hands-on expertise in...  ..., model training and evaluation, and numerical and scientific...  ...and reproducibility of ML and scientific work. Collaborate... 
    Contract work
    Part time
    Weekday work

    Mercor

    San Francisco, CA
    7 days ago
  • Adobe is looking for Machine Learning Engineer interns to work on generative AI features,...  ...intelligent agents, search, and production ML models used by hundreds of millions of people....  ...build and improve ML systems, train and evaluate models across the lifecycle, and... 
    Internship

    Adobe

    San Francisco, CA
    5 days ago
  •  ...Job Description Job Description AL/ML Engineer Santa Clara, CA FT Role This...  ...seeking to build and deploy machine learning models at scale, and thrives in an environment...  ...from fine tuning to building agents and evaluation pipelines. Collaborate with engineers... 
    Local area

    Info Way Solutions LLC

    San Francisco, CA
    21 days ago
  • $70 per hour

     ...across 15+ U.S. states. Software Engineering builds the brains of Waymo's...  ..., and losses to improve evaluation quality for the hillclimbing...  ...and evaluating deep learning models in a modern framework (JAX, PyTorch...  ..., plus a solid grounding in ML fundamentals: precision/... 
    Hourly pay
    Full time
    Internship
    Summer internship
    Relocation package

    Waymo

    San Francisco, CA
    5 days ago
  • $84 per hour

     .... Position: Atomistic Materials Modeling Expert Type: Contract Compensation:...  ...hours/week Role Responsibilities Evaluate advanced scientific problems involving...  ...Materials Science , Materials Science and Engineering , Computational Chemistry ,... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    7 days ago
  • $150 per hour

     ...Position: Machine Learning Expert Type: Contract...  ...machine learning engineering tasks for a long-horizon...  ...that challenge frontier models, ensuring they are solved...  ...Focus on inference, evaluation, and systems work rather...  ...or recent role as an ML research engineer , research... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $175k - $250k

     ...creating cutting-edge AI/ML products that solve...  ...within our AI Research & Engineering team. Operating at the...  ...allocation, long-context evaluation, and high-fidelity tool...  ..., execution logs, and expert labels into ultra-high-...  ...training datasets for model fine-tuning.Cross-Functional... 
    Flexible hours

    Ellipsis Health

    San Francisco, CA
    4 days ago
  • Title: ML Engineer Location: San Francisco, CA (Onsite) Direct HirePosition Summary:We're hiring...  ...Engineer to own the systems our models run on. You'll be working on the models...  ...run: reproducible training, trustworthy evaluation, clean packaging, versioned data and models... 
    Immediate start

    Spectraforce Technologies

    San Francisco, CA
    4 days ago
  •  ...are now building our Silicon Valley engineering team — a small, senior group focused...  ...reasoning loops. • Own prompt engineering, model versioning, evaluation (RAGAS, DeepEval), and LLMOps...  ...or related field. • 3–5 years of AI/ML engineering; minimum 2 years building... 
    Work at office
    Visa sponsorship

    Hop

    San Francisco, CA
    29 days ago
  •  ...Our client is a well-funded AI startup building production-grade ML infrastructure used by enterprise customers. They are looking for a Senior AI/ML Engineer to own model training pipelines, evaluation systems, and inference serving at scale. Full-time, on-site in San... 
    Full time

    Clera

    San Francisco, CA
    17 days ago
  • $65 - $68 per hour

     ...-generation Machine Learning (ML) platform for low-cost, ultra-...  ...scale data. Our Machine Learning Engineers are excited to work on these...  ...the problem, develop and evaluate new methods on real data, and...  ...Design, build, train and test ML models in areas such as reinforcement... 
    Summer work
    Internship
    Work at office
    Local area
    3 days per week

    Lyft

    San Francisco, CA
    1 day ago
  • $181k - $250k

     ...looking for someone that can build out the evaluation foundation across all of these products...  ...agentic products. You'll work on voice models, agent capability evals, benchmark...  ...related field ~3+ years of experience in ML Engineering or Applied ML with 8+ years of overall... 
    Contract work
    Worldwide

    Aircall.io, Inc.

    San Francisco, CA
    3 days ago
  • $150k - $250k

     ...environments or graders, and hardware engineering doesn't scare you. Be the first ML hire on data nobody else has. A...  ..., you convert rubrics set by experts and recordings of real CAD and ECAD...  ...into training environments, scoring models and reward signals, and package... 
    Full time
    Work at office
    Relocation
    Visa sponsorship

    RnB Consultancy

    San Francisco, CA
    6 days ago
  •  ...About the Role At Rippling, Engineering is at the heart of our...  ...crisp mathematical formulations (models); train those models; and,...  ...engineering, model training, and evaluation. You will work with data...  ...with the latest research in ML and related fields, and apply... 
    Full time
    Internship
    Work at office
    3 days per week

    Rippling

    San Francisco, CA
    5 days ago
  •  ...Perplexity is seeking experienced ML engineers to design, build, and optimize the recommendation...  ...and business metrics. Build user modeling that captures intent, preference, and...  ...experience for the user. Build the data and evaluation foundations that let these systems... 
    Full time

    Perplexity

    San Francisco, CA
    8 days ago
  • $180k - $240k

     ...of its clients a Senior Machine Learning Engineer - this is a fully remote role for US/Canada...  ...experience to join our small but mighty ML team building production-grade AI voice...  ...exploration and application of Large Language Models and Generative AI, venturing into new... 
    Remote work
    Flexible hours

    Career Renew

    San Francisco, CA
    27 days ago
  • $190k - $222k

     ...fusion. You will partner with robotics, controls, and hardware engineers to deliver reliable performance in real-world field conditions...  ..., tracking, classification, and sensor fusion. Build ML models and training pipelines, then deploy models to embedded hardware... 

    Clera

    San Francisco, CA
    5 days ago
  •  ...seeking a Machine Learning Infrastructure Engineer to help architect the compute, training,...  ...execution frameworks powering next-generation model performance. Backed by top-tier venture...  ...and optimize resource efficiency.Evaluate cutting-edge developments in model optimization... 
    Full time
    Work at office
    Flexible hours

    Objective Paradigm

    San Francisco, CA
    16 hours ago
  • $232k

     ...for a Staff Machine Learning Engineer to help define and build the next...  ...matching optimization and ML systems.Design matching objectives...  ...learning, probabilistic modeling, deep learning, causal inference...  ...causal measurement frameworks to evaluate policies and understand long-... 
    Full time
    Temporary work
    Work at office
    Remote work

    Uber

    San Francisco, CA
    5 days ago
  • $227.33k - $312.58k

    We’re looking for a Staff ML Data Engineer to join Procore’s AI & Frontier Models organization. In this role, you’ll be responsible for designing and building...  ...This is an opportunity to shape how Procore builds, evaluates, and deploys frontier models by ensuring the... 
    Full time
    Work at office
    Local area
    Immediate start
    3 days per week

    Procore Technologies

    San Francisco, CA
    2 days ago
  •  ...About Us Preference Model is a superintelligence data research company. We build...  ...go. We are looking for Senior ML Infrastructure Engineers to build the infrastructure and...  ...reducing time from idea to result Build evaluation and benchmarking infrastructure,... 
    Full time
    Visa sponsorship
    Relocation package

    Preference-Model

    San Francisco, CA
    4 days ago
  • $110k - $150k

     ..., technologists, and markets experts with deep roots in New York,...  ...that are actively hiring AI/ML Engineers . What We’re Looking For:...  ...across the full stack, from model integration to product-facing...  ...products Working on model evaluation, fine-tuning, and inference optimization... 
    Full time

    Dragonfly

    San Francisco, CA
    6 days ago
  •  ...patients. The Role As a Senior AI/ML Engineer at Condor, you will own the design and...  ...you will build, train, and serve custom models where they outperform general-purpose...  ...primitives—tool interfaces, retrieval systems, evaluation harnesses, and serving infrastructure—... 
    Full time
    Live in
    Immediate start

    condor-software

    San Francisco, CA
    4 days ago
  • $241k - $320k

     ...control problems, bringing modern ML to a domain traditionally...  ...methods. Develop learned models of vehicle behavior and dynamics...  ...a team of multidisciplinary Engineers and Research Scientists using...  ...Build the data pipelines, evaluation metrics, and tooling needed to... 

    Waabi

    San Francisco, CA
    29 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to ML Engineer - Model Evaluation Expert. Be the first to apply!