ML Engineer - Model Evaluation Expert
$60 - $90 per hourFull-time
Mercor
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Machine Learning Engineer — Model Evaluation & Experimentation
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week
Role Responsibilities
- Design tasks by transforming real ML research ideas into well-defined, multi-step tasks.
- Run experiments by implementing changes, executing training experiments, and analyzing results to define correct solutions.
- Explore reinforcement learning concepts such as reward functions and training behavior in task development.
- Evaluate frontier models' performance on tasks and identify areas of improvement.
- Collaborate with researchers to ensure tasks are consistent, rigorous, and fair.
Qualifications
Must-Have
- MSc or PhD in machine learning , computer science , or a related STEM field.
- 1+ years in a research or research-engineering role.
- Experience in training and evaluating ML models and conducting end-to-end experiments.
- Proficiency in Python and Git .
- Strong understanding of large language models and their evaluation.
Preferred
- Basic knowledge of reinforcement learning .
- Experience in AI training, model evaluation, or benchmark/task authoring.
Compensation & Legal
- Hourly contractor
- Paid weekly
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on jobs.jobcopilot.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 13 hours ago
Similar jobs that could be interesting for youBased on the ML Engineer - Model Evaluation Expert in Remote vacancy
$60 - $90 per hour
...Dorsey . Position: Machine Learning Engineer — Model Evaluation & Experimentation Type:... ...Design tasks by turning real ML research ideas into well-defined, multi... ...Collaborate with researchers and experts to maintain task consistency and rigor...SuggestedFull timeContract workSummer workRemote workMercor
Remote13 hours ago$144.7k - $261.3k
Job DescriptionAbout the TeamThe Evaluation Foundations team, part of Embodied AI's Scaling... ...for autonomous vehicle development. We engineer high-performance tools that identify top-performing models and partner with data-intensive ML teams to drive rapid innovation.Why...SuggestedFull timeLocal areaRemote workWork from homeRelocation packageFlexible hours- ...ML Engineer - LLM Evaluation & Automation We are seeking a highly skilled Machine Learning Engineer who specializes in leveraging Large Language Models (LLMs) for automated evaluation and quality assessment. In this role, you will design and build systems that automatically...SuggestedWork at officeRemote workFlexible hours
Grid Dynamics
United States3 days ago $213k - $263k
...ground for the Waymo Driver. The Simulator Evaluation team faces the ultimate data challenge:... ...are seeking visionary machine learning engineers and researchers to architect the... ...measure the realism of our multimodal world models. Your work will define the state of the...SuggestedFull timeRemote workWaymo
San Francisco, CA13 hours ago- 10a Labs is seeking a Machine Learning Engineer to design, build, and evaluate advanced ML systems across AI safety and model evaluation applications. You will tackle problems... ...analysts, red teamers, and subject‑matter experts to advance frontier AI, with focus on...SuggestedRemote job
- General Motors, through Embodied AI, seeks a Senior Engineer to measure and visualize AV model performance. You will design and implement evaluation workflows, collaborate across Data, Infra and validation teams, and connect offline evaluation with real-world behavior....Remote job
$204k - $259k
...Driver Understanding and Evaluation (DUE) team at Waymo is... .... It will combine expert human judgements and advanced... ...machine learning models to deliver training and... ...researchers and software engineers who are passionate about... ...implement scalable and robust ML pipelines for training,...Full timeWaymo
Remote13 hours ago$238k - $302k
...from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation. This role follows a hybrid... ...to a Senior Staff Software Engineer. You will: Work with... ...~5+ years of experience in ML engineering and applied Deep Learning...Full timeRemote workWaymo
Remote13 hours ago$251k - $310k
...improve and speed up the evaluation and onboard developer... ...journeys. It will combine expert human judgements and... ...advanced machine learning models to deliver training and... ...and software engineers who are passionate about... ...leadership on large-scale ML model architectures, especially...Full timeWaymo
Remote13 hours ago$195k - $300k
...Built by a world-class team: Engineers, designers, and operators from... ...— it's the foundation. As an ML Engineer, you'll be working at... ...intersection of cutting-edge model development and real-world legal... ...data curation and fine-tuning to evaluation and production deployment —...Full timeTemporary workWork at officeLocal areaFlexible hoursEve
Remote13 hours ago$177.3k - $212.8k
...teams use them to train various models, mapping teams use them to... ...latest developments in AI and ML for autonomous driving, 3D reconstruction... ...a customer-centric manner. Evaluate and make recommendations... ...consensus. Mentors and guides engineers within the group. Bachelor’...Full timeWork at officeImmediate startRelocationTorc Robotics
Remote13 hours ago$213k - $263k
...are currently focusing on include reinforcement learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation. This role follows a hybrid work schedule and reports to a Principal Research Scientist....Full timeTemporary workRemote workWaymo
New York, NY13 hours ago$199.2k - $298.8k
...teams use it to train various models and simulation teams use it for... ...latest developments in AI and ML for autonomous driving.... ...a customer-centric manner. Evaluate and make recommendations regarding... ...consensus. Mentors and guides engineers within the group. Bachelor...Full timeImmediate startRelocationCompany
Remote13 hours ago$175k - $200k
...Senior Machine Learning Engineer Truveta is the world... ...clinician to be an expert, and help families make... ...fine-tuned foundation models that continuously learn... ...engineering. You’ll blend ML craftsmanship with platform... .... Understand evaluation deeply — fluent in ML validation...Full timeFor contractorsVisa sponsorshipWork visaFlexible hoursTruveta
Remote13 hours ago- ...Machine Learning Engineer (Llama AI Platform) Location: Remote (Preferred... ...by open-source large language models, agentic workflows, and... ...workflows. Fine-tune and evaluate LLM performance for business use... ...tuning open-source LLMs. ML Engineering and MLOps...Full timeRemote work
Performacentric
Indianapolis, IN13 hours ago - ...specialize in physics-informed ML and enterprise AI solutions... ...Role: We are seeking an ML Engineer who builds ML systems directly... ...multiple systems — and ends with a model running on a schedule inside... ...and cleaning, modeling, evaluation, deployment into the client environment...Full timeRemote workWork visaFlexible hours
Azx Inc
Remote13 hours ago - ...science. We build foundational understanding of models to advance the frontier of intelligence. About the role: As a ML Engineer, you’ll build and operate the... ...work on the systems that support training and evaluating large models, scaling experimental pipelines...Full timeInternship
Tilde Research
Remote13 hours ago $200k - $300k
...preparation configurations. As a Senior ML Engineer, Manipulation, you will own the learning... ...parallel jaw, multi-finger) Implement and evaluate modern policy architectures (diffusion policies, transformer-based action models, action chunking) and adapt them to Chef's...Full timeFlexible hoursChef Robotics
Remote13 hours ago$213k - $263k
...are currently focusing on include reinforcement learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation. This role follows a hybrid work schedule and you will report to a Research Director of AI...Full timeRemote workWaymo
Remote13 hours ago- ...Responsibilities: Own evaluation pipelines — design, build, and... ...keep our speech and multimodal models honest in production. Harness... ...Required Qualifications: Expert-level PyTorch. Proven software engineer who loves ML; comfortable writing production...Full timeContract workFlexible hoursShift work
Sesame, L.l.c.
San Francisco, CA13 hours ago $174k - $253k
...implement robust agents and LLM-powered journeys that evaluate, gate, and improve engineering artifacts.Engineer and refine skills and context provided... ...experience.5 years of experience with Large Language Modeling (LLM).3 years of experience with generative AI Agents.Preferred...$100.4k - $180.7k
Posting TitleML and Optimization Engineer.LocationCO - Golden.Position... ...scientists, engineers, and experts are accelerating energy... ...high‑performance computing, AI/ML, modeling and simulation, and... ...end development and red‑team evaluation. NLR collaborates across DOE...Full timeFixed term contractLive inLocal areaRemote workRelocationShift work- ...work spans cutting-edge LLMs / ML, large-scale data systems,... ...joining a team of exceptional engineers, analysts, and investors working... ...inform how you think about modeling, signals, and product design.... ...model training to inference, evaluation, and production operations....Full timeWork at officeLocal area
Versant Limited
Remote13 hours ago $175k - $215k
...learning, learning from demonstration, generative modeling, Bayesian inference, hierarchical learning, and robust evaluation. In this hybrid role, you will report to a... ...industrial or research setting developing recipes for ML models We prefer: Track record of...Full timeRemote workWaymo
New York, NY13 hours ago$220k - $247k
...As a Senior Staff Machine Learning Engineer , you will operate at the company level—... ...You will lead the design of large-scale ML systems and shared platforms that power all... ...design of scalable ML platforms (training, evaluation, inference, safety) used across multiple...Full timeWork at officeImmediate startFlexible hours3 days per weekTypeface
Remote13 hours ago$85 per hour
...Summers , and Jack Dorsey . Position: ML Engineer (Coding Agent Experience) Type:... ...frontier AI coding agents to complete and evaluate complex machine learning and AI engineering tasks. Review model-generated implementations involving model training...Full timeContract workSummer workRemote workMercor
Remote13 hours ago$266k - $372.4k
...opportunity to quickly build a new kind of search engine. As a Senior Staff, you will design and... ...components. You will also deploy ML models, integrate LLMs, and ensure robust... ...working with LLM in production, including evaluation, tuning and deployment. ~ In-depth knowledge...Full timeFor contractorsWork experience placementRemote workFlexible hoursReddit
United States13 hours ago$153.2k - $234.1k
...behavior across real-world scenarios. As a Senior ML Infra Engineer, you will work on the core systems that enable rapid dataset generation, training, evaluation and iteration of our most advanced Autonomous Driving models. From enabling large foundational driving models...Full timeLocal areaRemote workWork from homeRelocation packageFlexible hours$189.3k - $320.7k
...vehicle behavior across real-world scenarios.As a Staff ML Engineer on the Prometheus team within the Embodied AI... ...estimates for multiple partner teams, including onboard model teams, simulation, and evaluation. The models you build will influence every stage of autonomous...Full timeLocal areaRemote workWork from homeRelocationRelocation packageFlexible hours$185k - $228k
...solve our customers most complex challenges. We’re seeking an experienced AI/ML engineer to work across the full analytical workflow—from data acquisition and model development to pipeline evaluation, application prototyping, and hardware-based solution implementation....Full timeRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to ML Engineer - Model Evaluation Expert. Be the first to apply!
Related searches
- ai ml engineer Remote
- graduate machine learning engineer Remote
- junior machine learning research engineer Remote
- junior machine learning engineer Remote
- senior ml engineer Remote
- machine learning ai engineer Remote
- computer vision machine learning engineer Remote
- data scientist machine learning engineer Remote
- machine learning engineer Remote
- machine learning software engineer Remote