Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Technical Lead - Autonomy Evaluation

$185k - $242k

ATOMS Careers page

Who we are 

Atoms is building the machines that power the next era of progress.

Over the last decade, software has transformed the digital world. But the physical world, where food is made, minerals are mined, goods are moved, and industries are run, remains far less intelligent, far less efficient, and far more constrained. We’re changing that.

Atoms builds Physical AI— real-world robots for the industries that move civilization forward, starting with food, mining, and transport. Our systems are designed to understand, predict, and control the real world with precision, turning complex physical operations into something more reliable, more scalable, and more productive.

This work requires more than robotics. It requires deep integration across hardware, software, AI, operations, manufacturing, and real estate. We don’t just build machines in a lab. We deploy them into real environments, operate them, learn from them, and improve them until they work at scale.

We are roboticists, engineers, operators, and builders. We believe the next great technology companies will not only transform information, but the physical systems that shape everyday life.

If you want to work on hard problems with real-world impact, join us.

 

About the role

We are seeking a Technical Lead for Autonomy Evaluation to own the metrics our autonomy releases are judged on and the log replay and evaluation platform that computes them. In this role, you will take evaluation from recorded logs, to scored regression suites running on every candidate release, to the report a release passes before it reaches a vehicle, developing how replay, scoring, metrics, and test coverage come together into one platform engineers use every day. You will do this across on-road vehicles, and you will build the evaluation infrastructure that lets each new software release, sensor configuration, and platform be assessed seamlessly.

 

What you’ll do

  • Metrics. Own the safety and behavior metrics a release is judged on, including collision and near-miss measures, trajectory agreement against ground truth, and comfort, and the go/no-go release criteria built on them. 
  • Log replay and scoring. Own the pipeline that converts recorded logs into scored test cases, covering open-loop and closed-loop evaluation of perception, localization, and planning against ground truth.
  • Regression and benchmarking. Own the system that benchmarks new software against old across the log corpus for every candidate release, including evaluation dataset design, regression detection, and triage of a regressed case to an owner.
  • Test coverage and data mining. Own how the corpus is mined for the long tail of rare and important events, including sampling strategy, selection of logs by expected value per replay-hour, and coverage of scenario classes the corpus does not yet contain.
  • Execution at scale. Own deterministic and reproducible execution of evaluation, run identity, traceability of every result to a software version, and cost per replay-hour as a tracked number.
  • Engineering practices. Set the practices for metric definitions, test design, code review, and the write-up of evaluation results.
  • Technical standard. Mentor the engineers who join the function, set the technical bar for their work, and participate in hiring.

 

What we’re looking for

  • Bachelor's degree in Computer Science, Robotics, Electrical Engineering, Statistics, or a related field with 8+ years of relevant experience.
  • 2+ years as a technical lead or engineering manager for an evaluation, validation, or test team.
  • Ownership of an offline or system-level evaluation system for an autonomous vehicle or robotics program, or a comparable large-scale ML model evaluation system in production, with accountability for the metrics and for the release decisions made on them.
  • Experience designing safety or behavior metrics for an autonomous system and defending them to engineering and executive audiences.
  • Hands-on experience building log replay at scale, including scoring against ground truth in open-loop and closed-loop evaluation.
  • Experience running evaluation pipelines on GPU clusters or comparable large-scale compute on a recurring cadence, with responsibility for reproducibility, throughput, and compute cost.
  • Experience with evaluation dataset design, benchmarking, and regression detection across software or model versions.
  • Working command of applied statistics for system validation, including statistical uncertainty, sampling, rare-event analysis, and the data volume a given claim requires.
  • Working understanding of autonomous systems end to end, including sensing, perception, localization, planning, and controls.
  • Experience on an autonomous vehicle or robotics platform.
  • Working experience with a robotics middleware and its logging and replay tooling.
  • Experience handling autonomous driving or robotics sensor data across multiple modalities and timestamps, including camera, LiDAR, and radar.
  • Proficiency in Python and the ability to read, navigate, and debug existing C++ codebases.

 

Why join us 

At Atoms, you’ll work on one of the defining challenges of our time—bringing automation into the physical world to drive real, lasting impact. We exist to uncover valuable unknown truths and turn them into progress, which means constantly pushing beyond what’s known and building what doesn’t yet exist. The work is ambitious and often challenging, but it’s grounded in a shared sense of purpose and a team committed to seeing it through together. Our work only matters if it serves others, and we know that meaningful progress depends on the trust of the people we serve and the strength of our team—so we invest in both, creating an environment where you can do your best work and grow.

 

What else you need to know 

This role is based in our San Francisco office location. As a company driven by innovation and continuous change, close collaboration is essential. We’re constantly reimagining our industry, creating new products, and refining our processes, and we do our best work together. That’s why all of our office-based teams work onsite, five days a week. 

The base salary range for this role is $185,000 - $242,000 per year.

Actual compensation will be determined on an individual basis and may vary depending on experience, skills, and qualifications.

Base salary is just one part of your total rewards package. You may also be eligible for equity awards.

Benefits Summary (USA Full-Time Exempt Employees):

  • Medical, Dental, Vision, Disability, and Life Insurance
  • Flexible Spending Account / Health Savings Account Options
  • 401(k)
  • Equity
  • Sick Time, Unlimited Flexible Time Off, and Paid Holidays
  • Paid Parental Leave 
  • Pre-Tax Commuter Benefit Plan

Benefits are subject to change at the company's discretion.
Atoms accepts applications on an ongoing basis.

 

Ready to join us as we serve those who serve others? 

#LI-Onsite

Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Technical Lead - Autonomy Evaluation in San Francisco, CA vacancy
  •  ...organization to deliver production-ready L4 autonomy across trucks, buses, passenger vehicles...  ..., and world models. As a founding technical leader, you'll define the architecture,...  ...deployable production systems. Shape safety, evaluation, and validation strategies. Build and... 
    Suggested
    Full time

    42dot

    San Francisco, CA
    2 days ago
  •  ...organization to deliver production-ready L4 autonomy across trucks buses passenger vehicles...  ...and world models. As a founding technical leader youll define the architecture research...  ...production systems. Shape safety evaluation and validation strategies. Build and... 
    Suggested
    Full time

    42dot

    San Francisco, CA
    11 hours ago
  • $250k - $350k

     ...continuous product self-improvement loops. As the Tech Lead Manager of AI Evaluations, you will lead a team of applied AI scientists and...  ...outcomes. You will: Own the AI evaluation eng roadmap and technical direction end to end: from defining what success means... 
    Suggested
    Work at office
    Home office
    Flexible hours

    Glean Work

    San Francisco, CA
    12 days ago
  • $75 - $105 per hour

     ...evidence behind your conclusion. You’ll work with a leading AI research organization to evaluate how coding agents perform on software engineering...  ...~ Clear written and spoken English for explaining technical findings and the evidence behind them. Nice to Have... 
    Suggested
    Hourly pay
    Immediate start
    Remote work

    Braintrust

    San Francisco, CA
    4 days ago
  • $180k - $260k

     ...environments, the ability to validate and prove our autonomy performance at scale becomes absolutely essential.We’re looking for a deeply technical software engineer to join the Autonomy...  ...understand how system behavior should be evaluated.Systems and data engineers, to ensure... 
    Suggested
    Local area

    Zipline

    South San Francisco, CA
    1 day ago
  • $153.98k - $230.98k

     ...Its products include Hivemind autonomy software, V-BAT and X-BAT...  ...validate customer use cases, evaluate the ease of use for autonomy...  ...platform bring-up and validation: Lead the integration of autonomy...  ...and presenting to customers or technical leadership.   Preferred... 
    Full time
    Temporary work
    Part time
    Worldwide

    Shield

    San Francisco, CA
    8 days ago
  •  ...About the Role We are hiring Sr. and Staff Software Engineers (Autonomy) to build the onboard and platform software that carries our...  ..., recording, and labeling data for perception development and evaluation ~ Build deployment, monitoring, and fault-management... 
    Full time

    marianaminerals

    San Francisco, CA
    8 days ago
  • $198.6k - $292k

     ...talk to you!We're hiring a Planning & Controls Engineer in our Autonomy Software team. In this role, you'll own and develop flight...  ...hardware testingMS/PhD degree in EE, ME, CS, Robotics, or related technical fieldNice to have experience includes:Familiarity with PX4... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    1 day ago
  •  ...Android: IOS: About the Role We are looking for a Technical Lead to drive technical execution across backend, frontend,...  ...needs, role requirements, interview criteria, and technical evaluation standards Build internal AI workflows and foster an AI-native... 
    Full time
    Remote work

    sekai

    San Francisco, CA
    8 days ago
  •  ...Apollo, and Clari. Come join us if you want to be part of a high autonomy, high pace team reinventing one of the biggest categories in enterprise software. The Role We're looking for a Technical Lead to own the technical direction of Monaco's Data Ingestion Team.... 
    Full time
    Work at office

    monaco

    San Francisco, CA
    8 days ago
  • $127k - $223k

     ...way. To learn more visit: The Evaluation Algorithms team is responsible for building...  ...to comprehensively evaluate our autonomy system’s performance across all development...  ...Science, Machine Learning and/or similar technical field(s) of study. Proficient in Python... 
    Full time
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    a month ago
  • $13 per hour

     ...up your career at the company leading workforce transformation in...  ...You'll work on one of the most technically challenging and impactful...  ...confidently at enterprise scale. Wide autonomy, real ownership, and...  ...accurately and reliablyCritically evaluate code (human or AI-generated)... 
    Full time

    Salesforce

    San Francisco, CA
    20 hours ago
  • $13 per hour

     ...up your career at the company leading workforce transformation in...  ...original company. You get the autonomy of a small team, backed by Salesforce...  ...workflows autonomouslyLead Technical StrategyPartner with Product...  ...in model evaluation. You have a track record of designing... 
    Full time

    Salesforce

    San Francisco, CA
    20 hours ago
  •  ...Abby Care is building the leading AI-native platform for family...  ...challenges of our time. Applied AI Technical Lead The Opportunity...  ...design, implementation, evaluation, launch, and iteration. Establish...  ...and strong ownership. AI autonomy is governed by explicit... 
    Full time
    Work experience placement
    Relocation

    abby-care

    San Francisco, CA
    3 days ago
  • $124k - $280k

     ...ManagerJob Description & SummaryThe OpportunityAs a Guidewire Technical Lead you will guide client work focused on Guidewire platform...  ...proprietary assets, or collaborating closely with team members. We evaluate these factors thoughtfully to establish a secure and trusted... 
    Full time
    H1b

    PwC

    San Francisco, CA
    2 days ago
  • $99k - $232k

     ...OpportunityAs a Guidewire Configuration Lead you will apply consulting skills to help...  ...Guidewire software implementations- Reviewing technical deliverables, identifying system...  ...collaborating closely with team members. We evaluate these factors thoughtfully to establish a... 
    Full time
    H1b

    PwC

    San Francisco, CA
    1 day ago
  • $298k - $368k

     ...Waymo Driver. We develop industry-leading simulation solutions using advanced...  ...ensure that Waymo's Simulation and Evaluation system seamlessly supports next-generation autonomy advancements and fleet growth, this Principal technical leadership role will look across multiple... 
    Full time
    Remote work

    Waymo

    San Francisco, CA
    5 days ago
  • $267k - $331.9k

     ...allows science and small teams of experts to lead the way. We build bridges to...  ...collaborations, and business development evaluations.Build strong relationships with external...  ...responsibility in Pharmaceutical Sciences, CMC, or Technical Operations.Experience leading... 
    Full time
    Work at office
    Flexible hours

    BridgeBio Pharma

    San Francisco, CA
    4 days ago
  •  ...Francisco, California | Primarily On-site We are seeking an AI Evaluation Engineer – Reinforcement Learning & Agents to build the...  ...engineering fundamentals. Demonstrated ability to build and ship technical infrastructure. Understanding of evaluation methodology,... 

    MaxIT Consulting - Max Corporate Group

    San Francisco, CA
    7 days ago
  • $200k - $300k

     ...collateral. Work with the sales team to write and build materials for technical audiences. Present proposals live to technical stakeholders....  ...behavior or interpretability work, such as post-training, evaluations or answer quality. Experience at a frontier AI lab or at a... 
    Full time

    Raydar

    San Francisco, CA
    1 day ago
  • $165k - $225k

    Backed by leading Silicon Valley investors, Peregrine helps public safety organizations, state and local and governments...  ...audiences ranging from public safety operators to federal evaluators to commercial technical buyers. The work starts from technical depth; everything... 
    Work experience placement
    Local area

    Peregrine Technologies

    San Francisco, CA
    1 day ago
  • $238k - $302k

     ...S. states. The Driver Understanding and Evaluation team at Waymo develops a rich understanding...  ...to an Engineering Manager. You will: Lead a top-tier applied ML team focused on building...  ...travel millions of miles. Drive technical direction, and provide technical inputs... 
    Full time
    Remote work

    CapitalG

    San Francisco, CA
    2 days ago
  • $230k - $390k

    About usSierra is the leading platform for customer-facing AI agents...  ...outcomes - we have a high-autonomy culture and each team member...  ...databases.Leadership experience on technical projects or teams.Our...  ...job description. We strive to evaluate all applicants consistently without... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    4 days ago
  • $230k - $390k

    About usSierra is the leading platform for customer-facing AI agents...  ...outcomes - we have a high-autonomy culture and each team member...  ...databases.Leadership experience on technical projects or teams.Our...  ...job description. We strive to evaluate all applicants consistently without... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    3 days ago
  • $148.5k - $223.9k

     ...up your career at the company leading workforce transformation in...  ...original company, offering the autonomy of a small team backed by the...  ...(SMTS), you will be a senior technical member of staff within our...  ...help our recruiters assess and evaluate candidates’ resumes and qualifications... 
    Full time
    Shift work

    Salesforce

    San Francisco, CA
    2 days ago
  • $168k - $247k

     ...of shipping production-level autonomy systems. The engineers will design...  .... You will be a key technical voice, expected to solve our...  ...urban/suburb driving scenarios.Lead root-cause analysis and systematically...  ...tail failures.Build scalable evaluation, simulation, and validation... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    1 day ago
  • $144k - $180k

    Who We AreAeroVect is transforming ground handling with autonomy, redefining how airlines and ground service providers around the globe...  ...and maintain radar data collection, labeling, and evaluation pipelinesArchitect a deterministic secondary perception system... 

    AeroVect

    San Francisco, CA
    20 hours ago
  • $230k - $390k

    About usSierra is the leading platform for customer-facing AI agents...  ...a bias to action in a high-autonomy environment.Degree in Computer...  ...record of leading complex technical projects or mentoring other engineers...  ...description. We strive to evaluate all applicants consistently... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    4 days ago
  •  ...platform indexes, curates, annotates, and evaluates data across the full AI lifecycle, from...  ...engineering-first role: FDSEs are the technical owners who take the Encord platform into...  ...technical stakeholdersOperate with high autonomy and own problems end to endFeed deployment... 
    Work at office

    Encord

    San Francisco, CA
    4 days ago
  • $96.8k - $306.4k

     ...languages: EnglishWe are seeking a Lead Principal Engineer to help shape the technical direction of Oracle’s AI-powered...  ..., development environments, and evaluation—to useful developer experiences.Make...  ...the appropriate balance of agent autonomy and human accountability.Ability... 
    Temporary work
    Flexible hours

    Oracle Corporation

    San Francisco, CA
    20 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Technical Lead - Autonomy Evaluation. Be the first to apply!