Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Software Engineer, Model Evaluation

Full-time

Clera

Responsibilities

  • Design and build evaluation suites for manufacturing and supply-chain workflows.
  • Create tasks, reference answers, rubrics, graders, and quality checks with domain experts.
  • Train and fine-tune models using evaluation results to guide experiments.
  • Identify agent failures, improve production systems, and measure the impact of changes.
  • Help establish the team’s research and evaluation practices.

Requirements

  • At least 3 years of experience operating at a senior or staff level.
  • Hands-on experience designing benchmarks and evaluations, including tasks, reference solutions or rubrics, and quality checks.
  • Practical experience training and fine-tuning models rather than only using models through APIs.
  • Ability to translate knowledge from non-ML domain experts into measurable tasks.
  • Supply-chain, procurement, manufacturing, or early-stage startup experience is helpful.

Benefits

  • Equity is included in the compensation package.
  • Hybrid work arrangement in San Francisco, California.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer, Model Evaluation in San Francisco, CA vacancy
  • $220k - $320k

     ...and hosts specialized language models for companies that need...  ...end: distillation, training, evaluation, and planet-scale hosting....  ...well-funded ten-person team of engineers who work in-person in downtown...  ...has founded and run their own software companies. We are high-agency... 
    Senior
    Full time
    Work at office

    inference

    San Francisco, CA
    9 days ago
  • $250k - $300k

     ...other, come build with us at Crusoe.About This Role:The Senior Staff Software Engineer for the AI Model Lifecycle team will play a crucial role in building...  ..., and experiment management: versioning, lineage, evaluation, and reproducible fine-tuning at scale.What You’ll... 
    Senior
    Temporary work

    Crusoe

    San Francisco, CA
    1 day ago
  •  ...and deploying frontier models for developers and...  ...team of researchers, engineers, designers, and more,...  ...future! Why this role?Evaluation is critical to making...  ...measure LLM progress. As a Senior Research Scientist,...  ...about. You have strong software engineering skills.... 
    Senior
    Full time
    Work at office
    Remote work
    Flexible hours

    cohere

    San Francisco, CA
    3 days ago
  • $204k - $259k

     ...5+ U.S. states. The Planner Evaluation team works on one of the key...  ...improving the quality of the software that drives the car. We are...  ...experienced data-minded software engineers and data scientists to help...  ...analysis tools for rapid modeling and prototyping... 
    Senior
    Full time
    Remote work

    Waymo

    San Francisco, CA
    6 days ago
  •  ...Create and evaluate datasets to enhance large language model performance. About Us Based in San Francisco, California, Turing is the world’s...  ...pipelines, plus top AI researchers who specialize in software engineering, logical reasoning, STEM, multilinguality,... 
    Senior
    Part time
    For contractors
    Flexible hours

    Turing

    San Francisco, CA
    16 days ago
  •  ...Evaluate and refine AI-generated code for efficiency and reliability. About Us...  ...top AI researchers who specialize in software engineering, logical reasoning, STEM, multilinguality...  ...benchmarking, and advancing large language models, collaborating closely with... 
    Senior
    Part time
    For contractors
    Flexible hours

    Turing

    San Francisco, CA
    16 days ago
  • $127k - $223k

     ...learn more visit: The Evaluation Algorithms team is...  ...realistic closed-loop simulation engine built with the latest in generative...  ...a holistic understanding of model performance and enable the discovery...  ...programming and strong software engineering fundamentals with... 
    Senior
    Full time
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    1 day ago
  • $136.44k - $166.75k

     ...structured data, and run AI agents and models directly in their workflows. Over 200,...  ...workhere.You’ll build the datasets, evaluations, and systems that help close that gap....  ...area. You’ll work at the intersection of software engineering, biology, and frontier AI: finding... 
    Work at office
    Local area
    Monday to Friday
    Shift work

    Benchling

    San Francisco, CA
    1 day ago
  • $75 - $105 per hour

     ...document the evidence behind your conclusion. You’ll work with a leading AI research organization to evaluate how coding agents perform on software engineering tasks. A failing test might point to a bug in the agent’s code, an overly restrictive assertion, an unclear... 
    Hourly pay
    Immediate start
    Remote work

    Braintrust

    San Francisco, CA
    8 hours ago
  • $77k - $202k

     ...OpportunityAs a Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate, you will provide strategic guidance and insights...  ...proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and... 
    Senior
    Full time
    H1b

    PwC

    San Francisco, CA
    3 days ago
  •  ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other...  ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value... 
    Senior
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    San Francisco, CA
    1 day ago
  •  ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other...  ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value... 
    Senior
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    San Francisco, CA
    1 day ago
  • $175k - $250k

     ...TIMEPosted: 2026-05-13About the RoleWe’re seeking a strong Senior Backend Engineer to join our AI Evaluation team, focused on building the infrastructure and...  ...about developer experience, holds a high bar for software-quality and takes pride in building tooling that makes... 
    Senior
    Flexible hours

    Ellipsis Health

    San Francisco, CA
    8 hours ago
  •  ...technical company. Both founders are engineers, and engineers have an...  ...AI Engineer to own how we evaluate and continuously improve the...  ...and ship confidently. Drive model evaluation and selection: Continuously...  ...Looking For ~5+ years of software engineering experience, with... 
    Senior
    Full time
    Work at office

    furtherai

    San Francisco, CA
    9 days ago
  • $85 per hour

     ...Jack Dorsey . Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type: Contract...  ...Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  • $144k - $180k

     ...deploy, and monitor radar object detection models for ground handling environmentsFuse...  ...maintain radar data collection, labeling, and evaluation pipelinesArchitect a deterministic...  ...radar as a primary modalityMentor junior engineers on radar perception best practicesYou haveBachelor... 
    Senior

    AeroVect

    San Francisco, CA
    1 day ago
  •  ...intelligence. You will conduct rigorous comparative analyses, build evaluation systems, and develop frameworks that capture reasoning and...  ...training, post-training, and applied teams to translate insights into model improvements. The role emphasizes high agency in a fast-paced... 

    Reflection AI

    San Francisco, CA
    5 days ago
  • TypeSafe AI is seeking a safety researcher to examine model behavior and help evaluate safety concerns within the team. You’ll build evals, document findings, and contribute to responsible research practices. During the six-week period, you’ll set up access, meet the team... 

    TypeSafe AI

    San Francisco, CA
    5 days ago
  • $159.2k - $301.6k

     ...team provides the platform engineering and deployment backbone...  ...entire organization delivers software. We're seeking a Senior Software Development...  ...agentic ecosystem. Extend our Model Context Protocol (MCP)...  ...around them - evaluation, feedback loops, observability... 
    Senior
    Temporary work
    Local area
    Worldwide

    Adobe

    San Francisco, CA
    2 days ago
  • $170k - $240k

     ...architecture. Hence, we are growing the engineering team and looking for...  ...and implement new software features to support our fast...  ...distributed system problemsData model design, and API development experienceSQL...  ...data for the purposes of evaluating your candidacy for employment... 
    Senior
    Full time
    Work at office

    Sigma Computing

    San Francisco, CA
    8 hours ago
  • $159.8k - $235k

     ...reliable on-demand, logistics engine for delivery! We're looking...  ...About the RoleWe’re looking for Senior Backend Engineers to work on...  ...-functionally, as well as evaluating and executing on trade-offs to...  ..., Codex, Cursor) in the full software development lifecycle, including... 
    Senior
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    4 days ago
  • $172.5k - $260.1k

     ...future of Salesforce.About the roleSlack is looking for a Senior Software Engineer to own Android accessibility across the app — from Slack Kit...  ...intelligence (AI) tools to help our recruiters assess and evaluate candidates’ resumes and qualifications throughout the recruiting... 
    Senior
    Full time

    Salesforce

    San Francisco, CA
    3 days ago
  • $190k - $235k

     ...customer churn, and save engineering resources—allowing them...  ...AI and large language models (LLMs) to build net-new...  ..., with candidate evaluation based on a combination...  ...-7+ years of full-time software engineering experience (looking for senior+ experience)An ability to... 
    Senior
    Full time
    Work at office
    Home office

    Merge API

    San Francisco, CA
    4 days ago
  • $148k - $185k

     ...looking for passionate, driven engineers to build systems that...  ...computing for ML and forecasting models, and finding patterns and...  ...Fulfillment, and more.As a Senior Software Engineer on FORTOP's forecasting...  ..., backtesting and evaluation tooling, and the serving layer... 
    Senior
    Hourly pay
    Work at office
    Local area
    Shift work
    3 days per week

    Lyft

    San Francisco, CA
    3 days ago
  • $152.5k - $205k

     ...stakeholder.What you’ll be responsible forA Senior Software Engineer on this team builds and maintains...  ..., access controls, auditability, evaluation, monitoring, and human approval where...  ...design, SQL database design, schema modeling, and query optimization.Demonstrated... 
    Senior
    Flexible hours

    Circle

    San Francisco, CA
    8 hours ago
  • $170k - $205k

     ...non-negotiable. We are seeking seasoned Senior Software Engineers to design, build, and scale this platform: the fine-grained authorization model for Crusoe Cloud resources, enterprise...  ...product, design, security, and SRE to evaluate tools, frameworks, and customer needs,... 
    Senior
    Temporary work

    Crusoe

    San Francisco, CA
    3 days ago
  • $172.43k - $209k

     ...charged with fueling the future of Crusoe’s engineering muscle. We’re responsible for critical...  ..., safely, and efficiently. As a Senior Software Engineer you will contribute to large...  ...teamsTechnical Decision-Making: Expert ability to evaluate technical tradeoffs and understand how... 
    Senior
    Full time
    Temporary work
    Work at office
    Local area

    Crusoe

    San Francisco, CA
    2 days ago
  •  ...You'll be one of the first engineers on the team, building the systems...  ...on customer feedback. Build evaluation and monitoring for our AI systems...  ...tokenization, and secure sandboxes for model work. Must haves ~5+ years of professional software engineering experience, with a... 
    Senior
    Full time
    Shift work

    negotiate-ai

    San Francisco, CA
    9 days ago
  • $148.5k - $223.9k

     ...Salesforce Enterprise Security Engineering team designs and operates...  ...deliver cloud-scale security software across multiple public cloud...  ...maintain services in a full DevOps model: monitor, troubleshoot, and...  ...our recruiters assess and evaluate candidates’ resumes and qualifications... 
    Senior
    Full time

    Salesforce

    San Francisco, CA
    2 days ago
  • $164k - $227k

    About the roleAs a Senior Software Engineer on the AI Enablement team, you will build the internal platforms, primitives, and guardrails that...  ...in a standardized, observable, and governed wayEstablish evaluation, observability, and cost-management practices across AI systems... 
    Senior
    Full time
    Work at office
    Local area
    Remote work
    Flexible hours

    Chime

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Software Engineer, Model Evaluation. Be the first to apply!