AI-Driven DevOps Engineer & Model Evaluator
Mercor Inc
Mercor is partnering with a leading AI research lab to support a Frontier Code Agents project. You will help evaluate frontier AI coding models by performing infrastructure engineering tasks and reviewing model-generated implementations on cloud platforms, Kubernetes, and CI/CD tools. This sprint-based role involves evaluating bugs, edge cases, and failure modes with professional engineering judgment, applying real-world reliability concepts to production-scale systems. #J-18808-Ljbffr Mercor Inc
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the AI-Driven DevOps Engineer & Model Evaluator in San Francisco, CA vacancy
$400 per month
...partnering with a leading AI research lab to support a... .... Contributors help evaluate and improve frontier AI coding models through structured technical... ...realistic infrastructure engineering workflows and model evaluation... ...2+ years of professional DevOps, SRE, or Cloud...SuggestedMercor Inc
San Francisco, CA1 day ago$50 - $75 per hour
A leading tech company based in Australia is seeking an AI Model Evaluator on a contract basis. The role involves evaluating AI-generated responses, writing prompts, and providing justifications based on specific criteria. Ideal candidates will hold a Master's degree in...SuggestedHourly payContract work$85 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: iOS Engineer (Coding Agent Experience) Type:... ...AI coding agents to complete and evaluate complex engineering tasks. Review model-generated mobile application code...SuggestedContract workSummer workRemote workMercor
San Francisco, CA23 days ago$144k - $187k
...building and modernizing research infrastructure with a focus on AI integration. The ideal candidate holds an M.S. or Ph.D. in a quantitative... ...skills in Python. Responsibilities include developing tools for model delivery and collaborating with cross-functional teams. The...SuggestedFlexible hours$144k - $187k
...designing and building research infrastructure that integrates AI, developing tools for model development, and maintaining automated validation and... ...S. or Ph.D. in a quantitative field and strong software engineering skills in Python are required. The position offers a...Suggested- UnitedMasters, LLC is seeking a Vice President of Engineering to lead and scale our engineering org across backend, mobile, web, data, QA, and DevOps. You will champion AI-driven personalization, tooling, and automation while defining the agentic‑development methodology...
$150k - $240k
...DevOps EngineerTitle of Role: DevOps EngineerLocation: San Francisco, on-site or remoteCompany... ...space, particularly within API SDKs and AI-driven solutions. With a strong commitment to... ..., RDS, and Redshift.Collaborate with engineering teams to streamline CI/CD pipelines...Recruiting from Scratch
San Francisco, CA5 days ago- UnitedMasters is seeking a Vice President of Engineering to lead and scale our engineering org across backend, mobile, web, data, QA, and DevOps. The role centers on architectural vision, delivery excellence, and AI‑driven product and developer tooling for both consumer...
$140k - $180k
Dealops in San Francisco, California is hiring a Senior Software Engineer to build an AI-driven revenue optimization platform. You'll work across the entire tech stack, focusing on deal pricing solutions and tools for revenue strategy optimization. The role demands strong...- Compa is seeking a Senior Software Engineer in San Francisco to take ownership of software systems. You will work closely with Product... ...compensation intelligence in a competitive market. Join an innovative team in a collaborative and driven environment. #J-18808-Ljbffr Compa
- ...Senior DevOps Engineer Skyfire empowers AI to present verified identities, access essential services and process payments without human intervention... ...Postgres, Redis, GitHub, Render, AWS If you are driven, quick, and love to learn, experiment, share, & develop...Work at officeShift work
Skyfire
San Francisco, CA2 days ago - ...platform concepts into clear, actionable content for developers and AI‑driven tools. Responsibilities include shaping the content roadmap,... ...material, runbooks, and diagrams; collaborating with engineers; and measuring impact on developer productivity and onboarding...
- ...Veeva Systems, a mission-driven SaaS leader, is building an AI-first platform for life sciences, and seeks a Principal DevOps Engineer to help found the team and scale cloud infrastructure. You will own IaC, CI/CD, Kubernetes, and AWS deployments, mentor engineers,...Remote jobFlexible hours
- ...building the category-defining AI workflow automation platform... ...operators, Plenful is driven by a deep understanding of the... ...Plenful is hiring a Staff DevOps Engineer to join our engineering team... ...York. R&D roles follow a hybrid model, with two days per week in our...Full timeWork at officeLocal areaRemote workFlexible hours2 days per week
Plenful
San Francisco, CA3 days ago - ...Director of Developer Platform & Experience to own the execution of AI-driven development and the platform underneath it. You will lead end-... ...start. You will be measured by speed, confidence, and reduced friction for hundreds of engineers. #J-18808-Ljbffr AI Chopping Block
- ...Francisco-based embedded finance company, is seeking a Data Platform Engineer to design and build scalable data platforms and pipelines. You will work with a team focused on reliable data delivery, data-driven decisions, and secure, high-performance data architectures. The...Work at office
- Mercor is seeking experienced sales engineers to define grading criteria for pre‑sales deliverables and to score AI‑generated and human work samples with detailed written... ...plans, demos, proofs of concept, and evaluation plans. The role emphasizes clear written communication...
- ...a strategy and design professional to lead enterprise operating model initiatives in the United States. You will partner with boards and... ...design, client relationship management, storytelling, and data-driven insights, with a strong consulting background and cross-industry...
$144k - $164k
...Virginia Manager, Product Management, Gen AI Model Gateway At Capital One, we’re changing... ...experiences, protection, and decisions driven by machine learning and advanced analytic... ...business analysis, data science, or software engineering. Preferred Qualifications: Bachelor’s...Full timePart timeLocal area- ...partner with Infrastructure, Inference, Research, and Product Engineering to shape the systems powering thousands of engineers and researchers... ...toolchains, guiding productivity and governance in an AI-driven era. You will drive end-to-end developer experience, establish...
- ...accelerate our internal developer ecosystem. You will oversee developer experience, infrastructure, build/CI, testing, and governance for AI-driven development across North America, Europe, and Asia. You will lead managers of managers, shaping strategy and execution, and...
- ...of Developer Platform & Experience to own the execution of an AI-driven development lifecycle and the platform underneath it. You will... ...CD systems, and the testing frameworks that enable hundreds of engineers to move quickly with confidence and governance. This builder's...
- ...the Role We're hiring software engineers for the Model Behavior team to help shape how Perplexity’s AI products behave: the style of... ...an analytical, experiment-driven approach to solving challenging... .... Experience designing evaluations or benchmarks for AI systems....
$150k - $230k
...Veeva Systems is a mission-driven organization and pioneer in industry... ...The Role As a Principal DevOps Engineer for a new product within... ...team building our next major AI-driven platform—one that will... ...~ Experience with SQL, data modeling, and design Interviewing...Work at officeLocal areaWork from homeFlexible hours- Gong is seeking a Senior Analyst, Analytics engineering to empower our analytics community by building, maintaining... ...with Data Ops to develop a semantic layer and AI-driven self-service analytics, designing data models in Snowflake with dbt, and ensuring data quality through...
$15 - $20 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...tools . Generate high-quality human evaluation data by identifying response strengths, areas... ...and completeness of responses. Ensure model responses align with expected conversational...Contract workSummer workRemote workMercor
San Francisco, CA23 days ago$200k - $240k
Alchemy is seeking a Data Engineer to build and manage data models in Snowflake, enabling AI-driven decisions company-wide. The ideal candidate has over 6 years in data engineering with strong SQL skills and extensive experience in Snowflake. Responsibilities include creating...Flexible hours$218.5k - $288k
...Scientist specializing in Small Language Models and AI Training, you will lead research and... .... You will work closely with research, engineering, and product teams to advance model training... ...language models.Design, implement, and evaluate model training experiments to improve...Work at officeFlexible hours3 days per week- UnitedMasters is seeking a Vice President of Engineering to lead and scale an engineering... ...across backend, mobile, web, data, QA, and DevOps. The leader will drive architectural vision, delivery excellence, and an AI-enabled culture for both consumer subscription...
$180k - $225k
As a Software Engineer on the ML Infrastructure team, you will design... ...engineers to integrate and optimize models for production and research... ...to ensure a fair and thorough evaluation of all applicants.About Us:At... ...is to develop reliable AI systems for the world's most important...Full time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI-Driven DevOps Engineer & Model Evaluator. Be the first to apply!
Related searches
- ai ml engineer San Francisco, CA
- machine learning ai engineer San Francisco, CA
- ai developer San Francisco, CA
- ai engineer San Francisco, CA
- ai research engineer San Francisco, CA
- ai prompt engineer San Francisco, CA
- senior ai engineer San Francisco, CA
- ai engineer remote San Francisco, CA
- senior devops engineer remote San Francisco, CA
- devops engineer sre San Francisco, CA