Senior Software Engineer, Model Evaluation
Full-time
Clera
Responsibilities
- Design and build evaluation suites for manufacturing and supply-chain workflows.
- Create tasks, reference answers, rubrics, graders, and quality checks with domain experts.
- Train and fine-tune models using evaluation results to guide experiments.
- Identify agent failures, improve production systems, and measure the impact of changes.
- Help establish the team’s research and evaluation practices.
Requirements
- At least 3 years of experience operating at a senior or staff level.
- Hands-on experience designing benchmarks and evaluations, including tasks, reference solutions or rubrics, and quality checks.
- Practical experience training and fine-tuning models rather than only using models through APIs.
- Ability to translate knowledge from non-ML domain experts into measurable tasks.
- Supply-chain, procurement, manufacturing, or early-stage startup experience is helpful.
Benefits
- Equity is included in the compensation package.
- Hybrid work arrangement in San Francisco, California.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer, Model Evaluation in San Francisco, CA vacancy
$220k - $320k
...and hosts specialized language models for companies that need... ...end: distillation, training, evaluation, and planet-scale hosting.... ...well-funded ten-person team of engineers who work in-person in downtown... ...has founded and run their own software companies. We are high-agency...SeniorFull timeWork at office$250k - $300k
...other, come build with us at Crusoe.About This Role:The Senior Staff Software Engineer for the AI Model Lifecycle team will play a crucial role in building... ..., and experiment management: versioning, lineage, evaluation, and reproducible fine-tuning at scale.What You’ll...SeniorTemporary work- ...and deploying frontier models for developers and... ...team of researchers, engineers, designers, and more,... ...future! Why this role?Evaluation is critical to making... ...measure LLM progress. As a Senior Research Scientist,... ...about. You have strong software engineering skills....SeniorFull timeWork at officeRemote workFlexible hours
$204k - $259k
...5+ U.S. states. The Planner Evaluation team works on one of the key... ...improving the quality of the software that drives the car. We are... ...experienced data-minded software engineers and data scientists to help... ...analysis tools for rapid modeling and prototyping...SeniorFull timeRemote work- ...Create and evaluate datasets to enhance large language model performance. About Us Based in San Francisco, California, Turing is the world’s... ...pipelines, plus top AI researchers who specialize in software engineering, logical reasoning, STEM, multilinguality,...SeniorPart timeFor contractorsFlexible hours
- ...Evaluate and refine AI-generated code for efficiency and reliability. About Us... ...top AI researchers who specialize in software engineering, logical reasoning, STEM, multilinguality... ...benchmarking, and advancing large language models, collaborating closely with...SeniorPart timeFor contractorsFlexible hours
$127k - $223k
...learn more visit: The Evaluation Algorithms team is... ...realistic closed-loop simulation engine built with the latest in generative... ...a holistic understanding of model performance and enable the discovery... ...programming and strong software engineering fundamentals with...SeniorFull timeWork at officeWork from homeFlexible hours$136.44k - $166.75k
...structured data, and run AI agents and models directly in their workflows. Over 200,... ...workhere.You’ll build the datasets, evaluations, and systems that help close that gap.... ...area. You’ll work at the intersection of software engineering, biology, and frontier AI: finding...Work at officeLocal areaMonday to FridayShift work$75 - $105 per hour
...document the evidence behind your conclusion. You’ll work with a leading AI research organization to evaluate how coding agents perform on software engineering tasks. A failing test might point to a bug in the agent’s code, an overly restrictive assertion, an unclear...Hourly payImmediate startRemote work$77k - $202k
...OpportunityAs a Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate, you will provide strategic guidance and insights... ...proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and...SeniorFull timeH1b- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
$175k - $250k
...TIMEPosted: 2026-05-13About the RoleWe’re seeking a strong Senior Backend Engineer to join our AI Evaluation team, focused on building the infrastructure and... ...about developer experience, holds a high bar for software-quality and takes pride in building tooling that makes...SeniorFlexible hours- ...technical company. Both founders are engineers, and engineers have an... ...AI Engineer to own how we evaluate and continuously improve the... ...and ship confidently. Drive model evaluation and selection: Continuously... ...Looking For ~5+ years of software engineering experience, with...SeniorFull timeWork at office
$85 per hour
...Jack Dorsey . Position: DevOps / SRE / Cloud Engineer (Coding Agent Experience) Type: Contract... ...Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms...Contract workSummer workRemote work$144k - $180k
...deploy, and monitor radar object detection models for ground handling environmentsFuse... ...maintain radar data collection, labeling, and evaluation pipelinesArchitect a deterministic... ...radar as a primary modalityMentor junior engineers on radar perception best practicesYou haveBachelor...Senior- ...intelligence. You will conduct rigorous comparative analyses, build evaluation systems, and develop frameworks that capture reasoning and... ...training, post-training, and applied teams to translate insights into model improvements. The role emphasizes high agency in a fast-paced...
- TypeSafe AI is seeking a safety researcher to examine model behavior and help evaluate safety concerns within the team. You’ll build evals, document findings, and contribute to responsible research practices. During the six-week period, you’ll set up access, meet the team...
$159.2k - $301.6k
...team provides the platform engineering and deployment backbone... ...entire organization delivers software. We're seeking a Senior Software Development... ...agentic ecosystem. Extend our Model Context Protocol (MCP)... ...around them - evaluation, feedback loops, observability...SeniorTemporary workLocal areaWorldwide$170k - $240k
...architecture. Hence, we are growing the engineering team and looking for... ...and implement new software features to support our fast... ...distributed system problemsData model design, and API development experienceSQL... ...data for the purposes of evaluating your candidacy for employment...SeniorFull timeWork at office$159.8k - $235k
...reliable on-demand, logistics engine for delivery! We're looking... ...About the RoleWe’re looking for Senior Backend Engineers to work on... ...-functionally, as well as evaluating and executing on trade-offs to... ..., Codex, Cursor) in the full software development lifecycle, including...SeniorHourly payWork at officeLocal areaRemote workFlexible hours$172.5k - $260.1k
...future of Salesforce.About the roleSlack is looking for a Senior Software Engineer to own Android accessibility across the app — from Slack Kit... ...intelligence (AI) tools to help our recruiters assess and evaluate candidates’ resumes and qualifications throughout the recruiting...SeniorFull time$190k - $235k
...customer churn, and save engineering resources—allowing them... ...AI and large language models (LLMs) to build net-new... ..., with candidate evaluation based on a combination... ...-7+ years of full-time software engineering experience (looking for senior+ experience)An ability to...SeniorFull timeWork at officeHome office$148k - $185k
...looking for passionate, driven engineers to build systems that... ...computing for ML and forecasting models, and finding patterns and... ...Fulfillment, and more.As a Senior Software Engineer on FORTOP's forecasting... ..., backtesting and evaluation tooling, and the serving layer...SeniorHourly payWork at officeLocal areaShift work3 days per week$152.5k - $205k
...stakeholder.What you’ll be responsible forA Senior Software Engineer on this team builds and maintains... ..., access controls, auditability, evaluation, monitoring, and human approval where... ...design, SQL database design, schema modeling, and query optimization.Demonstrated...SeniorFlexible hours$170k - $205k
...non-negotiable. We are seeking seasoned Senior Software Engineers to design, build, and scale this platform: the fine-grained authorization model for Crusoe Cloud resources, enterprise... ...product, design, security, and SRE to evaluate tools, frameworks, and customer needs,...SeniorTemporary work$172.43k - $209k
...charged with fueling the future of Crusoe’s engineering muscle. We’re responsible for critical... ..., safely, and efficiently. As a Senior Software Engineer you will contribute to large... ...teamsTechnical Decision-Making: Expert ability to evaluate technical tradeoffs and understand how...SeniorFull timeTemporary workWork at officeLocal area- ...You'll be one of the first engineers on the team, building the systems... ...on customer feedback. Build evaluation and monitoring for our AI systems... ...tokenization, and secure sandboxes for model work. Must haves ~5+ years of professional software engineering experience, with a...SeniorFull timeShift work
$148.5k - $223.9k
...Salesforce Enterprise Security Engineering team designs and operates... ...deliver cloud-scale security software across multiple public cloud... ...maintain services in a full DevOps model: monitor, troubleshoot, and... ...our recruiters assess and evaluate candidates’ resumes and qualifications...SeniorFull time$164k - $227k
About the roleAs a Senior Software Engineer on the AI Enablement team, you will build the internal platforms, primitives, and guardrails that... ...in a standardized, observable, and governed wayEstablish evaluation, observability, and cost-management practices across AI systems...SeniorFull timeWork at officeLocal areaRemote workFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Software Engineer, Model Evaluation. Be the first to apply!
Related searches
- senior software design engineer San Francisco, CA
- software web developer San Francisco, CA
- senior software engineer San Francisco, CA
- software developer fintech San Francisco, CA
- software support engineer San Francisco, CA
- security software engineer San Francisco, CA
- software developer night shift San Francisco, CA
- software engineer co-op San Francisco, CA
- integration software engineer San Francisco, CA
- scientific software engineer San Francisco, CA






