ML Engineer - Model Evaluation Expert
$60 - $90 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Machine Learning Engineer — Model Evaluation & Experimentation
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week
Role Responsibilities
- Design tasks by transforming real ML research ideas into well-defined, multi-step tasks.
- Run experiments by implementing changes, executing training experiments, and analyzing results to define correct solutions.
- Explore reinforcement learning concepts such as reward functions and training behavior in task development.
- Evaluate frontier models' performance on tasks and identify areas of improvement.
- Collaborate with researchers to ensure tasks are consistent, rigorous, and fair.
Qualifications
Must-Have
- MSc or PhD in machine learning , computer science , or a related STEM field.
- 1+ years in a research or research-engineering role.
- Experience in training and evaluating ML models and conducting end-to-end experiments.
- Proficiency in Python and Git .
- Strong understanding of large language models and their evaluation.
Preferred
- Basic knowledge of reinforcement learning .
- Experience in AI training, model evaluation, or benchmark/task authoring.
Compensation & Legal
- Hourly contractor
- Paid weekly
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on us.fitly.work
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
- ...an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance... ...and safety teams to refine evaluation rubrics and improve model alignment. Must have 5+ years in AI Safety or related fields and...SuggestedRemote jobContract workFlexible hours
- ...construction veterans and world-class engineers to solve physical-world... ...Machine Learning Engineer: Evaluation Bedrock is bringing... ...experience evaluating complex ML systems deployed in the real... ...evaluation use cases. Leverage models to source rich annotations for...SuggestedFull timeFor contractors
$180k - $220k
...careers alongside a team of experts across energy,... ...Crusoe is hiring an AI Engineer for our Developer Relations... ...workflows. Test features and models before launch, make... ...a software engineer, ML engineer, applied AI engineer... ...retrieval, memory, and evaluation, using one or more...SuggestedTemporary work$184k - $272k
...and the operators and workflow engines that keep long-running... ...what the ecosystem is shipping, evaluate honestly, and make the case with... ...dashboards researchers trust, a real model registry, and lineage from... ...Build observability across the ML stack — utilization,...SuggestedFull timeLive inWork at officeLocal areaRemote workWork from homeFlexible hours$184k - $272k
...and efficiency. Comprehensively profile model runtime and memory to pinpoint performance bottlenecks. Identify and evaluate emerging technologies that can be adopted... ...techniques. Work with researchers and ML engineers on best-practices for optimal resource usage...SuggestedWork at officeWork from homeFlexible hours- ...Job title: ML Engineer / Researcher, Large Language Models WORK AUTHORIZATION Must be authorized to work in the U.S.; we are unable to sponsor visas... .... You will design experiments, build datasets and evaluation systems, and fine-tune and post-train models. You...Full timeWork at officeVisa sponsorship
$90 - $140 per hour
...advanced Large Language Models. 1\. Overview Join a... ...talented Machine Learning Engineers and Scientific Computing experts with hands-on expertise in... ..., model training and evaluation, and numerical and scientific... ...and reproducibility of ML and scientific work. Collaborate...Contract workPart timeWeekday work- Adobe is looking for Machine Learning Engineer interns to work on generative AI features,... ...intelligent agents, search, and production ML models used by hundreds of millions of people.... ...build and improve ML systems, train and evaluate models across the lifecycle, and...Internship
- ...Job Description Job Description AL/ML Engineer Santa Clara, CA FT Role This... ...seeking to build and deploy machine learning models at scale, and thrives in an environment... ...from fine tuning to building agents and evaluation pipelines. Collaborate with engineers...Local area
$70 per hour
...across 15+ U.S. states. Software Engineering builds the brains of Waymo's... ..., and losses to improve evaluation quality for the hillclimbing... ...and evaluating deep learning models in a modern framework (JAX, PyTorch... ..., plus a solid grounding in ML fundamentals: precision/...Hourly payFull timeInternshipSummer internshipRelocation package$84 per hour
.... Position: Atomistic Materials Modeling Expert Type: Contract Compensation:... ...hours/week Role Responsibilities Evaluate advanced scientific problems involving... ...Materials Science , Materials Science and Engineering , Computational Chemistry ,...Contract workSummer workRemote work$150 per hour
...Position: Machine Learning Expert Type: Contract... ...machine learning engineering tasks for a long-horizon... ...that challenge frontier models, ensuring they are solved... ...Focus on inference, evaluation, and systems work rather... ...or recent role as an ML research engineer , research...Contract workSummer workRemote work$175k - $250k
...creating cutting-edge AI/ML products that solve... ...within our AI Research & Engineering team. Operating at the... ...allocation, long-context evaluation, and high-fidelity tool... ..., execution logs, and expert labels into ultra-high-... ...training datasets for model fine-tuning.Cross-Functional...Flexible hours- Title: ML Engineer Location: San Francisco, CA (Onsite) Direct HirePosition Summary:We're hiring... ...Engineer to own the systems our models run on. You'll be working on the models... ...run: reproducible training, trustworthy evaluation, clean packaging, versioned data and models...Immediate start
- ...are now building our Silicon Valley engineering team — a small, senior group focused... ...reasoning loops. • Own prompt engineering, model versioning, evaluation (RAGAS, DeepEval), and LLMOps... ...or related field. • 3–5 years of AI/ML engineering; minimum 2 years building...Work at officeVisa sponsorship
- ...Our client is a well-funded AI startup building production-grade ML infrastructure used by enterprise customers. They are looking for a Senior AI/ML Engineer to own model training pipelines, evaluation systems, and inference serving at scale. Full-time, on-site in San...Full time
$65 - $68 per hour
...-generation Machine Learning (ML) platform for low-cost, ultra-... ...scale data. Our Machine Learning Engineers are excited to work on these... ...the problem, develop and evaluate new methods on real data, and... ...Design, build, train and test ML models in areas such as reinforcement...Summer workInternshipWork at officeLocal area3 days per week$181k - $250k
...looking for someone that can build out the evaluation foundation across all of these products... ...agentic products. You'll work on voice models, agent capability evals, benchmark... ...related field ~3+ years of experience in ML Engineering or Applied ML with 8+ years of overall...Contract workWorldwide$150k - $250k
...environments or graders, and hardware engineering doesn't scare you. Be the first ML hire on data nobody else has. A... ..., you convert rubrics set by experts and recordings of real CAD and ECAD... ...into training environments, scoring models and reward signals, and package...Full timeWork at officeRelocationVisa sponsorship- ...About the Role At Rippling, Engineering is at the heart of our... ...crisp mathematical formulations (models); train those models; and,... ...engineering, model training, and evaluation. You will work with data... ...with the latest research in ML and related fields, and apply...Full timeInternshipWork at office3 days per week
- ...Perplexity is seeking experienced ML engineers to design, build, and optimize the recommendation... ...and business metrics. Build user modeling that captures intent, preference, and... ...experience for the user. Build the data and evaluation foundations that let these systems...Full time
$180k - $240k
...of its clients a Senior Machine Learning Engineer - this is a fully remote role for US/Canada... ...experience to join our small but mighty ML team building production-grade AI voice... ...exploration and application of Large Language Models and Generative AI, venturing into new...Remote workFlexible hours$190k - $222k
...fusion. You will partner with robotics, controls, and hardware engineers to deliver reliable performance in real-world field conditions... ..., tracking, classification, and sensor fusion. Build ML models and training pipelines, then deploy models to embedded hardware...- ...seeking a Machine Learning Infrastructure Engineer to help architect the compute, training,... ...execution frameworks powering next-generation model performance. Backed by top-tier venture... ...and optimize resource efficiency.Evaluate cutting-edge developments in model optimization...Full timeWork at officeFlexible hours
$232k
...for a Staff Machine Learning Engineer to help define and build the next... ...matching optimization and ML systems.Design matching objectives... ...learning, probabilistic modeling, deep learning, causal inference... ...causal measurement frameworks to evaluate policies and understand long-...Full timeTemporary workWork at officeRemote work$227.33k - $312.58k
We’re looking for a Staff ML Data Engineer to join Procore’s AI & Frontier Models organization. In this role, you’ll be responsible for designing and building... ...This is an opportunity to shape how Procore builds, evaluates, and deploys frontier models by ensuring the...Full timeWork at officeLocal areaImmediate start3 days per week- ...About Us Preference Model is a superintelligence data research company. We build... ...go. We are looking for Senior ML Infrastructure Engineers to build the infrastructure and... ...reducing time from idea to result Build evaluation and benchmarking infrastructure,...Full timeVisa sponsorshipRelocation package
$110k - $150k
..., technologists, and markets experts with deep roots in New York,... ...that are actively hiring AI/ML Engineers . What We’re Looking For:... ...across the full stack, from model integration to product-facing... ...products Working on model evaluation, fine-tuning, and inference optimization...Full time- ...patients. The Role As a Senior AI/ML Engineer at Condor, you will own the design and... ...you will build, train, and serve custom models where they outperform general-purpose... ...primitives—tool interfaces, retrieval systems, evaluation harnesses, and serving infrastructure—...Full timeLive inImmediate start
$241k - $320k
...control problems, bringing modern ML to a domain traditionally... ...methods. Develop learned models of vehicle behavior and dynamics... ...a team of multidisciplinary Engineers and Research Scientists using... ...Build the data pipelines, evaluation metrics, and tooling needed to...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to ML Engineer - Model Evaluation Expert. Be the first to apply!
- computer vision machine learning engineer San Francisco, CA
- machine learning software engineer San Francisco, CA
- graduate machine learning engineer San Francisco, CA
- ai ml engineer San Francisco, CA
- machine learning engineer San Francisco, CA
- senior ml engineer San Francisco, CA
- junior machine learning research engineer San Francisco, CA
- machine learning ai engineer San Francisco, CA
- machine learning scientist San Francisco, CA
- machine learning part time San Francisco, CA




