AI-Driven DevOps Engineer & Model Evaluator
Mercor
Mercor is partnering with a leading AI research lab to support a Frontier Code Agents project. You will help evaluate frontier AI coding models by performing infrastructure engineering tasks and reviewing model-generated implementations on cloud platforms, Kubernetes, and CI/CD tools. This sprint-based role involves evaluating bugs, edge cases, and failure modes with professional engineering judgment, applying real-world reliability concepts to production-scale systems. #J-18808-Ljbffr Mercor
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI-Driven DevOps Engineer & Model Evaluator in San Francisco, CA vacancy
$85 per hour
...technical talent with leading AI research labs. Headquartered... ...Dorsey . Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience)... ...coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving...SuggestedContract workSummer workRemote workMercor
San Francisco, CAa month ago$400 per month
...partnering with a leading AI research lab to support a... .... Contributors help evaluate and improve frontier AI coding models through structured technical... ...realistic infrastructure engineering workflows and model evaluation... ...2+ years of professional DevOps, SRE, or Cloud...Suggested$50 - $75 per hour
A leading tech company based in Australia is seeking an AI Model Evaluator on a contract basis. The role involves evaluating AI-generated responses, writing prompts, and providing justifications based on specific criteria. Ideal candidates will hold a Master's degree in...SuggestedHourly payContract work$85 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: iOS Engineer (Coding Agent Experience) Type:... ...AI coding agents to complete and evaluate complex engineering tasks. Review model-generated mobile application code...SuggestedContract workSummer workRemote workMercor
San Francisco, CA21 days ago$144k - $187k
...building and modernizing research infrastructure with a focus on AI integration. The ideal candidate holds an M.S. or Ph.D. in a quantitative... ...skills in Python. Responsibilities include developing tools for model delivery and collaborating with cross-functional teams. The...SuggestedFlexible hours$144k - $187k
...designing and building research infrastructure that integrates AI, developing tools for model development, and maintaining automated validation and... ...S. or Ph.D. in a quantitative field and strong software engineering skills in Python are required. The position offers a...- UnitedMasters, LLC is seeking a Vice President of Engineering to lead and scale our engineering org across backend, mobile, web, data, QA, and DevOps. You will champion AI-driven personalization, tooling, and automation while defining the agentic‑development methodology...
- UnitedMasters is seeking a Vice President of Engineering to lead and scale our engineering org across backend, mobile, web, data, QA, and DevOps. The role centers on architectural vision, delivery excellence, and AI‑driven product and developer tooling for both consumer...
- ...Compa is seeking a Senior Software Engineer in San Francisco to take ownership of software systems. You will work closely with Product... ...compensation intelligence in a competitive market. Join an innovative team in a collaborative and driven environment. #J-18808-Ljbffr
Compa
San Francisco, CA21 hours ago $140k - $180k
Dealops in San Francisco, California is hiring a Senior Software Engineer to build an AI-driven revenue optimization platform. You'll work across the entire tech stack, focusing on deal pricing solutions and tools for revenue strategy optimization. The role demands strong...- ...Veeva Systems, a mission-driven SaaS leader, is building an AI-first platform for life sciences, and seeks a Principal DevOps Engineer to help found the team and scale cloud infrastructure. You will own IaC, CI/CD, Kubernetes, and AWS deployments, mentor engineers,...Remote jobFlexible hours
- ...environments. Experience building AI-powered solutions, MCP Servers,... ...Qualifications Experience with DevOps and Site Reliability Engineering (SRE) practices. Strong production... ...resolution. Ideal Candidate A self-driven engineer who can take ownership of...
Eitacies Inc
San Francisco, CAa month ago - ...platform concepts into clear, actionable content for developers and AI‑driven tools. Responsibilities include shaping the content roadmap,... ...material, runbooks, and diagrams; collaborating with engineers; and measuring impact on developer productivity and onboarding...
- ...Francisco-based embedded finance company, is seeking a Data Platform Engineer to design and build scalable data platforms and pipelines. You will work with a team focused on reliable data delivery, data-driven decisions, and secure, high-performance data architectures....Work at office
Highnote
San Francisco, CA21 hours ago - ...a strategy and design professional to lead enterprise operating model initiatives in the United States. You will partner with boards and... ...design, client relationship management, storytelling, and data-driven insights, with a strong consulting background and cross-industry...
$144k - $164k
...Virginia Manager, Product Management, Gen AI Model Gateway At Capital One, we’re changing... ...experiences, protection, and decisions driven by machine learning and advanced analytic... ...business analysis, data science, or software engineering. Preferred Qualifications: Bachelor’s...Full timePart timeLocal area- ...partner with Infrastructure, Inference, Research, and Product Engineering to shape the systems powering thousands of engineers and researchers... ...toolchains, guiding productivity and governance in an AI-driven era. You will drive end-to-end developer experience, establish...
- ...accelerate our internal developer ecosystem. You will oversee developer experience, infrastructure, build/CI, testing, and governance for AI-driven development across North America, Europe, and Asia. You will lead managers of managers, shaping strategy and execution, and...
- ...of Developer Platform & Experience to own the execution of an AI-driven development lifecycle and the platform underneath it. You will... ...CD systems, and the testing frameworks that enable hundreds of engineers to move quickly with confidence and governance. This builder's...
- ...the Role We're hiring software engineers for the Model Behavior team to help shape how Perplexity’s AI products behave: the style of... ...an analytical, experiment-driven approach to solving challenging... .... Experience designing evaluations or benchmarks for AI systems....
- ...Director of Developer Platform & Experience to own the execution of AI-driven development and the platform underneath it. You will lead end-... ...start. You will be measured by speed, confidence, and reduced friction for hundreds of engineers. #J-18808-Ljbffr AI Chopping Block
- Mercor is seeking experienced sales engineers to define grading criteria for pre‑sales deliverables and to score AI‑generated and human work samples with detailed written... ...plans, demos, proofs of concept, and evaluation plans. The role emphasizes clear written communication...
$150k - $230k
...Veeva Systems is a mission-driven organization and pioneer in industry... ...The Role As a Principal DevOps Engineer for a new product within... ...team building our next major AI-driven platform—one that will... ...~ Experience with SQL, data modeling, and design Interviewing...Work at officeLocal areaWork from homeFlexible hours- Gong is seeking a Senior Analyst, Analytics engineering to empower our analytics community by building, maintaining... ...with Data Ops to develop a semantic layer and AI-driven self-service analytics, designing data models in Snowflake with dbt, and ensuring data quality through...
$15 - $20 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...tools . Generate high-quality human evaluation data by identifying response strengths, areas... ...and completeness of responses. Ensure model responses align with expected conversational...Contract workSummer workRemote workMercor
San Francisco, CA21 days ago$200k - $240k
Alchemy is seeking a Data Engineer to build and manage data models in Snowflake, enabling AI-driven decisions company-wide. The ideal candidate has over 6 years in data engineering with strong SQL skills and extensive experience in Snowflake. Responsibilities include creating...Flexible hours$218.5k - $288k
...Scientist specializing in Small Language Models and AI Training, you will lead research and... .... You will work closely with research, engineering, and product teams to advance model training... ...language models.Design, implement, and evaluate model training experiments to improve...Work at officeFlexible hours3 days per week- UnitedMasters is seeking a Vice President of Engineering to lead and scale an engineering... ...across backend, mobile, web, data, QA, and DevOps. The leader will drive architectural vision, delivery excellence, and an AI-enabled culture for both consumer subscription...
$180k - $225k
As a Software Engineer on the ML Infrastructure team, you will design... ...engineers to integrate and optimize models for production and research... ...to ensure a fair and thorough evaluation of all applicants.About Us:At... ...is to develop reliable AI systems for the world's most important...Full time$130k - $196.5k
...service environment creation for engineering teams.Lead and collaborate... ...quality.Mentor engineers on DevOps best practices, helping teams... ...infrastructure technologies by evaluating options, building proofs of... ...processing tools, analytics, and AI/ML workloads across LiveRamp...Full timeWork from homeFlexible hoursNight shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI-Driven DevOps Engineer & Model Evaluator. Be the first to apply!
Related searches
- ai ml engineer San Francisco, CA
- machine learning ai engineer San Francisco, CA
- ai developer San Francisco, CA
- ai engineer San Francisco, CA
- ai research engineer San Francisco, CA
- ai prompt engineer San Francisco, CA
- senior ai engineer San Francisco, CA
- ai engineer remote San Francisco, CA
- senior devops engineer remote San Francisco, CA
- devops engineer sre San Francisco, CA