Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Software Engineer, Evaluation Infrastructure

$127k - $223k

Waabi

Job Description

Job Description

Waabi, founded by AI visionary Raquel Urtasun, is the leader in Physical AI. With a world-class team, we're unlocking the next era of autonomous transportation with technology that's powering commercial autonomous trucks and robotaxis. Waabi is backed by and partners with world leaders in AI, automotive, logistics, and deep tech.

With offices in Toronto, San Francisco, Dallas, and Pittsburgh, Waabi is growing quickly and looking for diverse, innovative and collaborative candidates who want to impact the world in a positive way. To learn more visit:

The Evaluation Algorithms team is responsible for building the algorithms & tooling required to comprehensively evaluate our autonomy system’s performance across all development stages. In this role, you will work closely with the systems & safety teams, responsible for defining the requirements & evaluation criteria, and simulation teams to leverage Waabi World, our highly realistic closed-loop simulation engine built with the latest in generative AI technologies to deliver the evaluation capabilities needed to support the safe development of the next generation of autonomous vehicles!

 

You will…

  • Develop the tooling, infrastructure, and pipelines to support complex statistical analyses of driving performance scale.
  • Implement metrics and tags to provide a holistic understanding of model performance and enable the discovery of interesting scenarios for training and evaluation.
  • Develop and maintain a high availability query service to enable low-latency analysis and curation over large volumes of metric and tag data
  • Work with large datasets from various sources including real world driving as well as Waabi World, our high-fidelity simulator.
  • Champion engineering excellence, ensuring high-quality, well structured and tested code.
  • Assist in project roadmap planning, prioritisation, and delivery.

 

Qualifications: 

  • MS/Bachelors degree with 2+ years of industry experience in Computer Science, Machine Learning and/or similar technical field(s) of study.
  • Proficient in Python programming and strong software engineering fundamentals with real-world experience writing high quality, well-structured, and well-tested code. 
  • Open-minded and collaborative team player with the willingness to help others.
  • Passionate about self-driving technologies, solving hard problems, and creating innovative solutions.

 

Bonus Points:

  • Experience in data processing pipelines, ETL pipelines, distributed computing.
  • Experience in building highly reliable and scalable web services
  • Understanding of cloud job orchestration, monitoring, and instrumentation best-practices.
  • Experience in evaluating complex ML models or self-driving software stacks. 

The US yearly salary range for this role is: $127,000 - $223,000 USD in addition to competitive perks & benefits. Waabi (US) Inc.’s yearly salary ranges are determined based on several factors in accordance with the Company’s compensation practices. The salary base range is reflective of the minimum and maximum target for new hire salaries for the position across all US locations. Note: The Company provides additional compensation for employees in this role, including equity incentive awards and an annual performance bonus.

Perks/Benefits:

  • Competitive compensation and equity awards.
  • Health and Wellness benefits encompassing Medical, Dental and Vision coverage (for full-time employees only).
  • Unlimited Vacation.
  • Flexible hours and Work from Home support.
  • Daily drinks, snacks and catered meals (when in office).
  • Regularly scheduled team building activities and social events both on-site, off-site & virtually.
  • As we grow, this list continues to evolve! 

Waabi is a technology start-up building technologies to transform the way the world moves. Join our talented team to be a part of the future and to make an impact!

Waabi is an equal opportunity employer. We celebrate diversity and are committed to creating a supportive, inclusive, and accessible workplace for all our employees. We seek applicants of all backgrounds and identities, across race, color, ethnicity, national origin or ancestry, age, citizenship, religion, sex, sexual orientation, gender identity or expression, military or veteran status, marital status, pregnancy or parental status, caregiver status, disability, or any other characteristic protected by law. We make workplace accommodations for qualified individuals with disabilities as required by applicable law. If reasonable accommodation is needed to participate in the job application or interview process please let our recruiting team know.

We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses and identifying potential inconsistencies or verification signals in application materials based on available information. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.

Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Senior Software Engineer, Evaluation Infrastructure in San Francisco, CA vacancy
  • $165k - $226.6k

     ...secures AI by building the trusted, neutral infrastructure that enables organizations to safely...  ...users’ experiences.  Okta's Core Engineering team is responsible for building and...  ...Redis, Docker, Terraform on AWS) Drive evaluation, development and rollout of new common... 
    Senior
    Permanent employment
    Work at office
    Local area
    Worldwide
    Flexible hours

    Okta

    San Francisco, CA
    4 days ago
  •  ...sustainable future. About the Role The Infrastructure Engineering team is crucial to the overall...  ...perform deep-dive code reviews, and evaluate technical trade-offs to ensure a sustainable...  ...: Establish and enforce elite software engineering and DevOps standards, including... 
    Senior
    Shift work

    Hayden AI

    San Francisco, CA
    2 days ago
  • $204k - $259k

    Senior Software Engineer, Quantitative Evaluations Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver...  ...Experience with ML Experience with A/B experiment infrastructure Experience building and validating metrics to... 
    Senior
    Full time
    Remote work

    Waymo

    San Francisco, CA
    2 days ago
  • $183k - $220k

     ...every 10 workers, but lacks good software tools to get the job done....  ...all of us. ???? The Role As a senior member of our small but mighty engineering team, you'll work closely with our...  ...evolution of our backend tech stack and infrastructure. Whether iterating on core... 
    Senior
    Full time
    For contractors
    Work at office
    Remote work

    Siteline

    San Francisco, CA
    17 days ago
  • What to expect This is a full-time on-site role located in San Francisco for a Senior Software Engineer, Infrastructure. You will be responsible for scaling our infrastructure, owning major architecture decisions, and building the platform on which product features are... 
    Senior
    Full time

    Orion Sleep

    San Francisco, CA
    2 days ago
  • $216k - $270k

     ...frontier AI labs build, deploy, and evaluate AI systems at scale.The Orchestration...  ...and customer-facing workflows.As a Senior Software Engineer, you will help build and evolve this...  ...systems, reliability, and platform infrastructure used by many internal teams.You will... 
    Senior
    Full time

    Scale AI

    San Francisco, CA
    1 day ago
  •  ...worldwide.We’re a team of engineers, clinicians, and innovators...  ...DescriptionPrimary Function of PositionAs a Senior Software Engineer - Simulation & ML...  ...development, sim-to-real evaluation, and machine-learning...  ...and maintain software infrastructure for robotic simulations &... 
    Senior
    Local area
    Worldwide
    Flexible hours

    Intuitive Surgical

    San Francisco, CA
    1 day ago
  • $180.5k - $236.91k

     ...Hi, we're Oscar. We're hiring a Senior Software Engineer, Cloud Infrastructure / SRE to join our Engineering team. Oscar is the first health insurance company built around a full stack technology platform and a relentless focus on serving our members. We started Oscar... 
    Senior
    Full time
    Work at office
    Remote work

    Oscar

    San Francisco, CA
    20 days ago
  • $216k - $270k

     ...Senior Software Engineer, Orchestration Platform Scale is building reliable AI systems for the...  ...frontier AI labs build, deploy, and evaluate AI systems at scale. The Orchestration...  ...systems, reliability, and platform infrastructure used by many internal teams. You... 
    Senior
    Full time

    Scale AI

    San Francisco, CA
    13 hours ago
  •  ...About us For institutions Press Login Meanwhile Senior Software Engineer - Platform, API Full-time · Engineering · San Francisco, CA (3...  ...~ Fluency with AI development tools, and the judgment to evaluate what they produce ~ Comfort with ambiguity and a habit of... 
    Senior
    Permanent employment
    Full time
    Contract work
    Work at office
    Weekend work
    3 days per week

    Meanwhile

    San Francisco, CA
    13 hours ago
  • Senior Software Engineer At Commure, we're building the AI Operating System for healthcare, the foundation that defines how care is delivered...  ...the Role We're hiring a Senior Software Engineer on the Infrastructure team to own the foundational infrastructure and internal... 
    Senior
    Local area
    Immediate start

    Commure

    San Francisco, CA
    2 days ago
  • $187 per hour

     ...team builds and operates the infrastructure, tooling, and developer...  ...enable data scientists and ML engineers to develop, train, deploy,...  ...reliably and efficiently. As a Senior Software Engineer on the Machine...  ..., foundation-model access, evaluation, and agentic orchestration.... 
    Senior
    Full time

    Chime

    San Francisco, CA
    23 days ago
  •  ...Job Description Job Description Senior Software Engineer, ML Platform Location: San Francisco...  ..., Feature Stores, Underwriting Infrastructure About Our Client Our client is...  ...behind model experimentation, training, evaluation, inference, and retraining is... 
    Senior
    Full time
    Remote work
    Flexible hours

    NxT Level

    San Francisco, CA
    a month ago
  • Software Engineer Voxel's perception system is the technical core of everything we ship. Our models detect human activity, equipment interactions...  .... We're hiring a strong software engineer to own the ML Infrastructure that powers how Voxel trains and ships vision models. You'... 
    Senior
    Work at office
    Flexible hours

    Voxel

    San Francisco, CA
    2 days ago
  • Platform Engineer HUD is building infrastructure to create RL training data and evals for frontier AI agents, as well as a marketplace to sell these to...  ...Can write clean, maintainable code and apply strong software engineering judgment across product architecture, infrastructure... 
    Senior
    Full time
    Work at office
    Remote work
    Relocation
    Visa sponsorship

    Hud (yc W25)

    San Francisco, CA
    2 days ago
  • $190k - $221k

    Senior Software Engineer, Infrastructure TRM Labs is a blockchain intelligence company committed to fighting crime and creating a safer world. By leveraging blockchain data, threat intelligence, and advanced analytics, our products empower governments, financial institutions... 
    Senior
    Remote work

    TRM Labs

    San Francisco, CA
    4 days ago
  • $176k - $220k

     ...researchers to create evaluations, publish benchmarks,...  ...Work together with engineers, scientists, operators...  ...Human data is the core infrastructure to AI advancement....  ...Handshake is hiring a Senior LLM Platform Engineer...  ...Strong production software engineering experience... 
    Senior
    Full time
    Work at office
    Remote work
    Flexible hours

    Handshake

    San Francisco, CA
    15 days ago
  • $188k - $235k

     ...Scale GP is building the infrastructure that makes enterprise AI...  ...seamless. We are looking for a Senior Infrastructure Engineer to act as a primary...  ...building, orchestrating, and evaluating multi-agent systems at...  ...with 5+ years of full-time software engineering experience.... 
    Senior
    Full time

    Scale AI

    San Francisco, CA
    3 days ago
  •  ...of sales into a scalable science. About the Role The Core Infrastructure team makes the rest of Nooks fast. We're going from one...  ...and reps spend 4+ hours a day in the product. We're hiring senior engineers to own the systems everything else is built on. What you'll... 
    Senior
    Work at office
    3 days per week

    Nooks

    San Francisco, CA
    2 days ago
  • $208k - $260k

     ...lab researchers to create evaluations, publish benchmarks, and push...  ...Work together with engineers, scientists, operators, and...  ...Human data is the core infrastructure to AI advancement. Frontier...  ...Role Handshake is hiring a Senior Software Engineer to join Backend Platform... 
    Senior
    Full time
    Work at office
    Remote work
    Flexible hours

    Handshake

    San Francisco, CA
    4 days ago
  • $124k - $329.2k

     ...leading platform for agentic software development - powered by...  ...looking for an experienced engineer to join GitHub's Identity organization...  ...perspectives to help evaluate emerging standards, guide architectural...  ...building identity infrastructure for enterprise, cloud, or developer... 
    Senior
    Ongoing contract
    Remote work

    GitHub

    San Francisco, CA
    2 days ago
  • $210k - $300k

     ...obsessed team of the world's best engineers and operators. If you are...  ...the Role We're looking for a Software Engineer to join our ML Infrastructure team. In this role, you'll help build...  ...reviews with peers and stakeholders to evaluate technical tradeoffs and select... 
    Senior
    Local area
    Flexible hours

    Nimble Robotics

    San Francisco, CA
    17 days ago
  • $200k - $260k

     ...are a hardcore and obsessed team of the world's best engineers and operators. If you are obsessed with your craft,...  ...agency. About the Role We are looking for a Senior Software Engineer to join our Infrastructure Team , focused on building secure, reliable, and... 
    Senior
    Local area
    Flexible hours

    Nimble Robotics

    San Francisco, CA
    22 days ago
  • $200k - $400k

     ...cloud and inside customers' own locked-down environments) is an infrastructure problem, and that's the problem this role owns. You'll build...  ...cloud infrastructure, Kubernetes, and networking that let engineers ship without becoming infrastructure experts. Make sure it all... 
    Senior
    Full time
    Work at office
    Local area
    Shift work

    Decagon

    San Francisco, CA
    2 days ago
  •  ...Handshake is hiring a Senior Software Engineer to build the systems powering high-quality coding data and evaluations for next‑gen AI models. You’ll work on complex projects...  ...execution to design reliable, scalable infrastructure that enables teams to create and improve... 
    Senior
    Remote job

    Jobleads-US

    San Francisco, CA
    1 day ago
  •  ...decarbonize their business. We're looking for software engineers to help build the AI platform that...  ...You Will: Design and build the agent infrastructure that powers Watershed's products...  ..., making it possible to debug, evaluate, and improve agent behavior at scale... 
    Senior
    Work at office
    Remote work

    WaterSHED

    San Francisco, CA
    2 days ago
  •  ...Job Description Job Description Senior Software Engineer – Backend Systems & Infrastructure Location: San Francisco - on-site Compensation: Competitive Base Salary + Equity + Benefits About the Opportunity We are partnering with a venture-backed technology... 
    Senior

    MRINetwork Jobs

    San Francisco, CA
    a month ago
  • $216k - $270k

     ...frontier AI labs build, deploy, and evaluate AI systems at scale. The Orchestration...  ...-facing workflows. As a Senior Software Engineer, you will help build and evolve this...  ...systems, reliability, and platform infrastructure used by many internal teams. You... 
    Senior
    Full time

    Scale AI

    San Francisco, CA
    3 days ago
  • $217.8k - $326.8k

    Senior Software Engineer Block builds simple, powerful tools that make progress towards an economy...  ..., distributed systems, cloud infrastructure, and developer tooling. This is a software...  ...Block is an equal opportunity employer evaluating all employees and job applicants... 
    Senior
    Work experience placement
    Local area
    Remote work
    Flexible hours

    Block | Square

    San Francisco, CA
    2 days ago
  • $216k - $270k

    Software is eating the world, but AI is eating software. We live...  ...fine-tuning, and production evaluations. This is the main innovation...  ...include the Generative AI Data Engine, SGP, Donovan, and others...  ...architecting Scale's core cloud infrastructure and orchestration stack, and... 
    Senior
    Full time
    Live in

    Scale AI

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Software Engineer, Evaluation Infrastructure. Be the first to apply!