Member of Technical Staff, Evals Lead
$2,000 per monthbuild-ai
About Build AI
Build AI is the data hyperscaler for Physical AI. We're vertically integrated across hardware, manufacturing, logistics, collection, and model training to scale the physical labor dataset orders of magnitude faster than anyone in the world.
Job Summary
We’re hiring an Evals Lead to benchmark video and world-model capabilities with and without our data. You need to deeply understand the philosophy of evals: what an eval is allowed to claim, contamination, leakage, construct validity, and whether a number actually corresponds to a capability. More practically, you should have pushed consequential evals before — evals that changed what a lab trained, shipped, or collected, not a weekend leaderboard.
Key Responsibilities
Design and run benchmarks for video and world models, with Build data and without it, under the same protocol
Make the comparison honest: held-out tasks, contamination and leakage checks, no cooking the numbers
Build the analytics layer: model performance, failure patterns, and whether scaling the dataset moves capability — and on which axes
Work with research and dataset/quality so collection and evals inform each other
Push evals that are consequential enough that people change plans when the number moves
Design processes that increase evaluation quality, repeatability, and scale as we add tasks and countries
You may be a good fit if you have (Must-have qualifications)
You have shipped or driven evals that mattered: they changed training, hiring, product, or data decisions
You understand eval philosophy well enough to argue about validity, not only to plot a curve
Ideally video, robotics, world models, or multimodal, but the bar is consequential evals more than a specific domain
You will not confuse a pretty dashboard with an eval that is allowed to decide things
Strong candidates may also have experience with (Nice-to-have qualifications)
Video, robotics, world models, or multimodal evals
You have designed evals used in a paper, a product launch, or a data decision
Background in construct validity, contamination, or leakage
Exposure to evaluation operations: throughput, failure taxonomy, repeatability
Benefits
Competitive pay
Medical, dental, and vision packages with generous premium coverage
$500 per month credit for waiving medical benefits
Housing subsidy of $2k per month for those living within walking distance of the office
Relocation support for those moving to San Francisco (Financial District) or Shenzhen (Nanshan)
Various wellness benefits covering fitness, mental health, and more
Daily lunch and dinner in our office
Unlimited compute budget subject to ROI justification
Unlimited Codex and Claude credits
Travel
How we're different
Build believes in the Bitter Lesson . By taking a general approach of learning from humans, our addressable market is all physical labor.
We are a fully in-person team in San Francisco (Financial District) and Shenzhen (Nanshan), and greatly value engineering skills. We do not have boundaries between engineering and research, and we expect all of our technical staff to contribute to both and work across disciplines as needed.
Build AI is an equal opportunity employer. We review every application. If you do not meet every bullet, still apply. Questions: View email address on jobs.ashbyhq.com
- ...Palantir—and help build the systems that make expert human knowledge useful for advancing AI. The Role We are hiring a Member of Technical Staff, Evals to help define how frontier AI systems are measured, understood, and improved. This is a broad, high-ownership role...SuggestedFull timeWork at officeFlexible hours
$225k
...this goal. About the role The Evals team measures the performance and capabilities... ...'s most important decisions. As a Member of Technical Staff on Evals, you will develop new... ...subtle bugs or flawed assumptions that lead to misleading conclusions. We care deeply...SuggestedRemote jobFull timeVisa sponsorshipRelocation package$150k - $220k
# Founding Member of Technical Staff (MTS)Bay Area, CAFull-time$150k-$220k + equity## About UsVizopsAI is the secure runtime for custom enterprise... ....## What You'll Do* •Build backend services for training, evals, telemetry, and online policy updates* •Instrument...Suggested- ...that runs order-to-cash for the world’s leading enterprises. Across order management,... ...ai . The Role We’re hiring a Member of Technical Staff – AI/ML to design, build, and deploy... ...calibration Have strong opinions on AI/ML evals — golden datasets, offline + online...SuggestedFull timeFlexible hours
- ...Member of Technical Staff @ Lotus AI Who we are Lotus AI is a groundbreaking primary care app that integrates your medical records, AI... ...and fine-tuning, model tooling, data pipelines, retrieval/evals, and product workflows. You'll be close to the core system...SuggestedFull time
- ..., and actionable. In this role, you will build and improve the technical foundations that support Answer Quality across Perplexity. This... ...shared evaluation infrastructure that helps teams run reliable evals, analyze results, and make product and model decisions Develop...Remote jobFull time
- ...evaluation infrastructure alongside the world’s leading AI labs. Now, we’re bringing those... ...experts, and devise new methods, so that evals and rubrics keep improving and so do the... ...Degree in Computer Science or a related technical degree such as Electrical Engineering, Physics...Full timeWork at officeLocal areaRelocation package
$180k - $200k
...Job Description Job Description Member of Technical Staff – Full Stack (Backend-Leaning) Location... ...agent evaluation, LLM systems, or evals/QA. This role may not be for you if... ...engineering, solutions, product, or management/lead role. This is a hands-on engineering...Full timeWork at officeImmediate start- ...Read more here, here, and here. The role You’ll lead the red-teaming of the world’s most used AI agents,... ...Refine the core engine that runs all our red-teaming evals DRI end-to-end: technical scoping, reconnaissance, execution, remediation, certification...RelocationVisa sponsorship
- ...Our AI agents do the research a great analyst would. They follow leads across company sites, public records, and social platforms,... ...done. New data sources and tools for the enrichment engine. Evals that tell us whether a prompt, model, or agent change made results...Full timeWork at officeNight shift
- ...Francisco or Seattle with a hybrid work arrangement, offering a collaborative environment where technical and product decisions shape the company's future. The Member of Technical Staff position involves full-stack product engineering, working closely with clinicians and...
- ...Member of Technical Staff – Machine Learning I’m partnering with a rapidly scaling healthtech startup that has just raised a $40M Series A to expand its engineering team. Their AI-powered platform is already helping clinicians by automating huge amounts of back-office...Work at office
- ...within a native office suite, and deploys them in secure environments for Fortune 500 companies. About the Role As a Member of Technical Staff, you will be part of the team responsible for the work platform. You will think end-to-end about what it means to build...Work experience placementH1bWork at officeVisa sponsorship
- ...applied across dozens of biopharma partnerships. Backed by leading technology and biotechnology investors including a16z, Lux,... ...machine learning and drug discovery. We are seeking a Member of Technical Staff, Machine Learning to join our talented, ambitious, and...
- ...AI coding tools such as Claude Code or Codex Clear thinking about technical trade-offs, and clear written communication Experience doing research or running rigorous experiments, such as LLM evals, benchmarks or A/B tests, and drawing sound conclusions from noisy...Full time
- ...OpenAI, Google DeepMind, Cursor, Decagon, CoreWeave, Profound, Northwood Space, Solidworks, and more. About the Role Member of Technical Staff is a generalist position where you will collaborate closely with the founders on product direction, implementation, and...
$227.5k - $401k
...Member of Technical Staff San Francisco This is Adyen Adyen provides payments, data, and financial products in a single solution for global... ...or fixing critical performance issues. Learn and Lead: Connect with the broader AI community (including startups,...Work at officeImmediate startRelocationFlexible hours- ...getting started. THE ROLE As a Member of the Technical Staff, you build the system that makes our agents... ...and remediate findings, and build the evals that tell us whether they actually got... ...Work with customers and our security leads to turn real findings, misses, and...Full time
- ...hyperscalers, and AI-native companies, giving our team access to technical problems spanning frontier models, production infrastructure, and emerging hardware. About the role As a Member of Technical Staff, you will build the systems that schedule, route, and...Full time
- ...Stuut is the AI coworker that runs order-to-cash for the world’s leading enterprises. Across order management, credit, collections,... ...finance teams set rules, review decisions and steer agents. Write evals against real customer outcomes, and turn each failure into a fix...Full timeFlexible hours
- ...them from idea through deployment. Collaborate on architecture, product decisions, and technical direction. Help define product engineering practices as an early team member. Requirements ~6+ years of experience with deeper architectural ownership and emphasis...Full timeTemporary workWork at officeRemote work
$200k - $300k
...Job Description Job Description Member of Technical Staff Company: Confidential — AI cloud on a mission to maximize intelligence per watt Location: San Francisco, CA (FiDi office, on-site 5 days per week) Compensation: $200,000 - $300,000 + 0.5% - 1% equity...Full timeInternshipH1bWork at officeOverseasVisa sponsorship- ...and enterprises that integrate LLMs into their products. The team is 5 people with a research and product focus. As a Member of Technical Staff on our infrastructure team, you'll own the cloud systems that serve our compression API end-to-end. You'd get to build...Full timeVisa sponsorship
$150k - $300k
...the mission. Be dependable - Take ownership and deliver with high agency. The Role Nimble is looking for a Member of Technical Staff to help us advance our robotics moonshot by designing, developing, training, and implementing robotic foundation models,...Local areaImmediate startFlexible hoursWeekend work- ...internal agents across government, Fortune 500, frontier labs and hyper-growth startups. Teams use our configurable primitives such as evals, observability, router, authentication, permissions, sandboxes, harnesses and more to deploy internal agents at scale securely. We...Full time
- ...Job Description Job Description About the Role We are hiring Members of Technical Staff internally referred to as Founding Engineers. The mandate is to architect and own the core systems that power real-time AI voice and compliance workflows for top-10 US mortgage...H1bImmediate startRelocationRelocation package
- ...Member of Technical Staff, Science Our mission is to advance humanity's ability to engineer biology. We approach this by understanding and improving AI’s application to biology across basic research, drug discovery, and biosecurity. We are a team of engineers, scientists...Full timeWork at officeVisa sponsorshipRelocation packageFlexible hours
- ...systems engineering - distributed systems, APIs, databases, cloud infra. Expert Python (Go or TypeScript a plus). Terraform, GCP, containers, multi-tenant production experience. Taste for the LLM-specific infra problems: model routing, evals, cost and latency controls....Full time
- ...good mental models of to do good work. Great interpersonal and technical communication. Please don't use LLMs to write prose. We desk-... ...you want to hear. # Share an online whiteboard with a team member and work through a technical problem. We spend a lot of time at...Full timeWork at office
$300k
...scale: distributed rollouts, training orchestration, inference, evals, data pipelines, observability, and reliability. You will... ...Experience building tools, platforms, or services used by other technical users. Strong judgment around technical tradeoffs: when to prototype...Work at officeLocal area
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Member of Technical Staff, Evals Lead. Be the first to apply!
- helpdesk support technician San Francisco, CA
- tech assistant San Francisco, CA
- technical support assistant San Francisco, CA
- technical analyst San Francisco, CA
- desktop support analyst San Francisco, CA
- user support analyst San Francisco, CA
- support technician San Francisco, CA
- product support technician San Francisco, CA
- mri tech aide San Francisco, CA
- technical support analyst San Francisco, CA



