DevOps Engineer - AI Model Evaluator - AI Trainer
$400 per monthMercor Inc
About the Role Mercor is partnering with a leading AI research lab to support a Frontier Code Agents project. Contributors help evaluate and improve frontier AI coding models through structured technical assessments. The work focuses on realistic infrastructure engineering workflows and model evaluation. Spots are limited and filling quickly on a first come, first serve basis. What You'll Do Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms, Kubernetes, CI/CD systems, observability, and infrastructure automation. Identify bugs, edge cases, reliability issues, and failure modes. Compare outputs from multiple frontier models and assess their strengths and weaknesses. Apply professional engineering judgment to realistic infrastructure engineering scenarios. Time Commitment Sprint based project that runs in 12-24 hour stretches based on client requirement. Compensation $400 per accepted task. Typical tasks take approximately 2–3 hours after ramp-up. Compensation is tied to accepted work. Who Should Apply 2+ years of professional DevOps, SRE, or Cloud Engineering experience. Experience with AWS, Azure, GCP, Kubernetes, Terraform, CI/CD pipelines, or observability tooling. Regular use of AI coding agents such as Cursor, Claude Code, Codex, Windsurf, Gemini CLI, or similar tools. Ability to evaluate model-generated infrastructure and reliability engineering solutions. Experience supporting production-scale systems is preferred. #J-18808-Ljbffr Mercor Inc
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the DevOps Engineer - AI Model Evaluator - AI Trainer in San Francisco, CA vacancy
$10 - $20 per hour
...Role Overview Help train next-generation robotic AI systems through precise, hands-on data collection in a San Francisco lab. You will... ...everyday manipulation tasks that support how robot learning models learn, reason, and perform. No prior robotics or AI experience is...SuggestedHourly payContract workSaidGig
San Francisco, CA4 days ago$30 per hour
A leading platform for AI training is seeking advanced Japanese speakers to join as Domain... ...complete various AI training tasks and evaluate AI responses in Japanese. Compensation is... ...a dynamic environment aimed at improving powerful AI models. #J-18808-Ljbffr ProlificSuggestedRemote jobHourly payWork from homeFlexible hours$30 per hour
AI Trainer - Advanced Japanese Fluency Prolific is building the biggest pool of quality human data in the world. Over 35,00... ...looking for advanced Japanese speakers to help train and evaluate cutting‑edge AI models. If you have the necessary experience, you’ll take a...SuggestedRemote jobHourly paySelf employmentWork from homeFlexible hours$170k - $216k
...of miles of driving data from a diverse set of sensors, enabling engineers like you to (1) develop methods for efficiently and continuously learning from large scale real-world data, to (2) develop models and model training at scale, to (3) analyze real-world behavior and...SuggestedFull timeRemote workWaymo
San Francisco, CA9 hours ago- ...critical inference for the world's most dynamic AI companies, like Cursor, Notion,... ...the frontier of AI to bring cutting-edge models into production. We're growing quickly and... ...Conviction. Join us and help build the platform engineers turn to to ship AI products. THE...SuggestedFull timeFlexible hours
Baseten
San Francisco, CA9 hours ago - ...enterprise and developers alike to use and access our start-of-the-art AI models, allowing them to do things that they’ve never been able to... ...model inference. About the Role We are looking for an engineer who wants to take the world's largest and most capable AI...Full time
OpenAI
San Francisco, CA9 hours ago $298k - $368k
...of miles of driving data from a diverse set of sensors, enabling engineers like you to (1) develop methods for efficiently and continuously learning from large scale real-world data, to (2) develop models and model training at scale, to (3) analyze real-world behavior and...Full timeRemote workWaymo
San Francisco, CA9 hours ago- ...About the Team We’re hiring software engineers to make OpenAI’s Model Performance teams more productive. These teams work on the systems, tooling,... ...with forward progress About OpenAI OpenAI is an AI research and deployment company dedicated to ensuring that...Full time
OpenAI
San Francisco, CA9 hours ago - ...OpenAI’s Hardware organization develops AI-native silicon and system-level solutions... ...to power the next generation of frontier models. By co-designing chips, systems, tools, and... ...build the model runtime within the inference engine that executes complex, frontier models at...Full time
OpenAI
San Francisco, CA9 hours ago - Welo Data is seeking Data Labeling Associates in California to evaluate AI outputs and ensure cultural context and safety in Arabic datasets. This role requires professional-level proficiency in Portuguese (Brazil), a bachelor's degree, and at least 2 years of experience...
- Obsidian is hiring expert Evaluators in Healthcare operations for a remote, hourly engagement. You will review AI-generated work products for accuracy, rigor, and domain quality, leveraging your extensive subject-matter expertise in the field. The ideal candidate has over...Remote jobHourly payWork at office
- ...Description Job Description DevOps Engineer Role Type: Contractor... ...DevOps Engineers to create and evaluate production-style infrastructure... ...tasks for next-generation AI systems. You will work on CI/... ...Previous experience with AI training data or model evaluation....For contractorsRemote work
YO AI Labs
San Francisco, CA22 days ago $160k - $250k
...creating cutting-edge AI/ML products that solve... ...for a seasoned DevSecOps engineer with significant experience... ...SLA Service cost modeling and ongoing cost... ...Configurations, etc. ~ Solid DevOps experience, preferably... ...applicable laws. We evaluate candidates holistically...Work at officeLocal areaFlexible hoursEllipsis Health
San Francisco, CA2 days ago- ...Description Job Description We are looking for a Senior Software Engineer, Infrastructure with ideally 5+ years of experience to own and... ...build, release, and runtime foundations at a fast-scaling voice AI company. You'll design and automate deployment pipelines across...
NxT Level
San Francisco, CA24 days ago - ...Kubernetes-based environments. • Experience building AI-powered solutions, MCP Servers, Agentic AI systems, or GenAI... .... Preferred Qualifications • Experience with DevOps and Site Reliability Engineering (SRE) practices. • Strong production support, incident...
Eitacies Inc
San Francisco, CA24 days ago $150k - $200k
Get AI-powered advice on this job and more exclusive features. Understanding Recruitment provided pay range This range is provided... ...working with a leading AI company, who are looking for a DevOps Engineer to join their ever-expanding Engineering team who can bridge the...Full timeWork at officeImmediate start2 days per week3 days per week$40 per hour
Overview AI Trainer - Advanced SQL Developers About Prolific Prolific is not just another player in the AI space - we are... ...role We’re looking for SQL Developers to help train and evaluate cutting-edge AI models. If you have the necessary experience, we’ll send you an...Remote jobSelf employmentWork from homeFlexible hours$183k - $248k
...out of hardware. To do this, we’re building the world's first AI hardware engineer. Founded in 2019, our platform enables anyone to go from... ...and that means everything around it has to just work. As a DevOps Engineer, you’ll work on the full-stack systems that power Flux...Local areaShift workFlux
San Francisco, CA22 days ago- Senior DevOps Engineer Stack AI is building cutting-edge infrastructure at the intersection of AI and enterprise. Our mission is to enable companies to safely and reliably deploy AI at scale. As we grow, we're looking for a Senior DevOps Engineer who can design, build,...Work at officeImmediate startRemote work
$145k - $165k
...Description Evolver is seeking a skilled DevOps Engineer who has transitioned into network... ...Terraform, Azure DevOps pipelines, and AI-assisted development tools such as Claude... ...of experience building AIOps pipelines, Model Context Protocol (MCP) integrations or LLM...Flexible hoursEvolver Commercial
San Francisco, CA24 days ago$220k - $320k
...trains and hosts specialized language models for companies that need frontier-quality AI at a fraction of the cost. The... ...-to-end: distillation, training, evaluation, and planet-scale hosting. We... ...a well-funded ten-person team of engineers who work in-person in downtown...Work at officeInference
San Francisco, CA4 days ago$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Role...Contract workSummer workWork at officeRemote work$150k - $180k
...is the universal data layer for AI that helps 300+ AI teams train and run models on the right data. Our platform... ...indexes, curates, annotates, and evaluates data across the full AI lifecycle... ...The role We're looking for a DevOps Engineer with 4–5 years of experience to...Work at officeFlexible hoursEncord
San Francisco, CA24 days ago- ...Join a leading AI lab's cutting-edge GenAI team to be at the... ...most advanced Large Language Models. 1. Overview Join a leading... ...re seeking talented Cloud and DevOps Engineers with deep, hands-on expertise... ...infrastructure automation problems. Evaluate infrastructure engineering...Full timeContract workWeekday work
Mercor
San Francisco, CA8 days ago $180k - $275k
...pay range $180,000.00/yr - $275,000.00/yr We’re building an AI-native fintech platform that’s modernizing how finance teams operate... ...can support complex financial workflows at speed. Position DevOps Engineer - Own and evolve the cloud infrastructure that underpins our...Full time$230k - $320k
DevOps Engineer At EliseAI, we're improving the industries that matter most: housing and healthcare. Everyone needs a place to live and access... ...often harder to secure than they should be. By integrating AI agents deeply into existing workflows, we make them more efficient...Work at officeLocal areaRelocation$153k - $202k
Join to apply for the DevOps Engineer role at Merge Merge is the leading provider of agentic tools and customer-facing integrations for frontier... ...with a single API, and Merge Agent Handler, which empowers AI agents with secure access to thousands of third-party tools....Full timeWork at officeHome office- ...business will answer their phone with voice AI. We are making sure that when they build... ...high-pressure environment to work in our engineering. This role is critical to the success of... ...upgrades. Qualifications: 5+ years as a DevOps engineer Experience writing async web apps...Full timeWork at office
$140k - $200k
DevOps Engineer San Francisco, CA About Pump.co Cloud spend is a whopping $500 billion/yr, the biggest growing expense category for any tech... ...is building the fastest way to save ~60% on cloud spend. Our AI-powered platform not only fully automates savings, but we also...Work at office- Senior DevOps Engineer Skyfire empowers AI to present verified identities, access essential services and process payments without human intervention. From API account creation to monetizing websites, we drive commerce for the world's fastest-growing consumer base: AI agents...Work at officeShift work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to DevOps Engineer - AI Model Evaluator - AI Trainer. Be the first to apply!
Related searches
- devops engineer San Francisco, CA
- big data devops engineer San Francisco, CA
- devops aws developer (remote) San Francisco, CA
- senior devops engineer remote San Francisco, CA
- senior devops cloud engineer San Francisco, CA
- senior devops engineer San Francisco, CA
- devops engineer full time San Francisco, CA
- education evaluator San Francisco, CA
- social media evaluator San Francisco, CA
- evaluator San Francisco, CA