Technical Lead, Evaluation Infrastructure
Full-time
Nuro
Who We Are Nuro is a self-driving technology company on a mission to make autonomy accessible to all. Founded in 2016, Nuro is building the world’s most scalable driver, combining cutting-edge AI with automotive-grade hardware. Nuro licenses its core technology, the Nuro Driver™, to support a wide range of applications, from robotaxis and commercial fleets to personally owned vehicles. With technology proven over years of self-driving deployments, Nuro gives the automakers and mobility platforms a clear path to AVs at commercial scale, empowering a safer, richer, and more connected future. About The Role Evaluation Infrastructure plays a critical role at Nuro, directly enabling L4 driverless deployment. The team supports two demanding workloads: day-to-day Autonomy Evaluation that powers rapid software iteration, and large-scale Driverless Safety Validation that produces the rigorous evidence required to deploy autonomy on public roads. The Evaluation Infrastructure team builds the metrics framework, evaluation pipelines, introspection tooling, and analysis products that turn raw on-road and simulation logs into actionable insight. Our metrics stack spans both heuristic and ML-based approaches, covering everything from low-level component accuracy to end-to-end behavior quality. The platform empowers autonomy and Systems & Safety teams to run complex evaluations and validations across a wide range of configurations and scales, producing the high-fidelity metrics that drive both short-term iteration and long-term release confidence — in close partnership with Simulation and the broader AI Platform. As the Technical Lead, you will lead the team with deep technical guidance and rigor, setting the technical bar, shortening the time-to-signal for evaluation and the time-to-confidence for validation, so that both autonomy and Systems & Safety teams can iterate fast while deploying software safely. About The Work
- Build and own a unified metrics, evaluation, and validation platform — pipelines, introspection tooling, and analysis products that turn on-road and simulation logs into high-fidelity signals for autonomy iteration and driverless safety validation
- Drive the technical bar for metric quality across both heuristic and ML-based approaches; invest in the scale, reliability, and CI/CD of the evaluation stack to shorten time-to-signal for evaluation and time-to-confidence for validation, and to meet high SLAs for downstream stakeholders
- Mentor and grow the Evaluation Infrastructure team, and champion AI-native engineering practices that compound team velocity and code quality
- Partner with Product, Autonomy, Systems & Safety, and Simulation teams to define and execute the vision and strategy for evaluation at Nuro
- You have a degree in B.Sc or M.Sc., plus 4 years of relevant work experience
- Domain experience: Strong fluency in distributed systems, large-scale data and ML evaluation pipelines, metrics frameworks (heuristic and/or ML-based), and analytics platforms
- Engineering leadership: Experience setting technical vision, roadmap, and prioritization for a team operating at the intersection of autonomy, safety, and data infrastructure; a clear, concise communicator who partners effectively with PMs, engineers, and cross-functional stakeholders across Autonomy, Systems & Safety, and Simulation
- Technical excellence: Ability and willingness to deep-dive into implementation; sets the technical bar for metric quality, pipeline rigor, and safety-critical engineering practice across the broader software organization; strong proficiency in Python, C++, or similar languages
- AI-native mindset: Daily user of modern AI coding assistants and agentic tools (Claude Code, Cursor, and similar), with strong intuition for where they accelerate engineering work and where they don't; eager to apply LLMs and ML systems to evaluation problems, from automated triage and metric generation to natural-language analysis of fleet behavior; raises the team's productivity, code quality, and signal density through thoughtful AI integration
- Knowledge of data engineering, and its tooling and best practices
- Knowledge of batch and streaming data processing, warehousing, and analytics solutions
- Experience with data workflow orchestration platforms
- Prior experience building evaluation, validation, or analytics platforms, ideally in autonomy, robotics, or safety-critical systems
Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Technical Lead, Evaluation Infrastructure in California vacancy
$184k - $259.44k
...motivated Software Engineer, Frontier AI Infrastructure to join our dynamic Public Sector... ...usage metrics for AI models. You will lead technical discussions on the frontlines with cloud... ...us to ensure a fair and thorough evaluation of all applicants. About Us:...SuggestedFull timeWork at office3 days per weekEarly shift$127k - $223k
...positive way. To learn more visit: The Evaluation Algorithms team is responsible for... ...You will… - Develop the tooling, infrastructure, and pipelines to support complex statistical... ...Science, Machine Learning and/or similar technical field(s) of study. - Proficient in...SuggestedFull timeWork at officeWork from homeFlexible hours$235.03k - $352.29k
...Fidelity, T. Rowe Price, and other leading investors. About the Role... ...Leader to drive the technical roadmap for the systems that... ...will lead the development of evaluation tooling that ensures our technology... ...with Autonomy and Infrastructure engineers to set a roadmap that...SuggestedImmediate startFlexible hours$175k - $215k
...ground for the Waymo Driver. The Simulator Evaluation team faces the ultimate data challenge:... ...key components of our data processing infrastructure. You will write and maintain the... ...Iterate: You will work closely with Tech Leads to refine our evaluation capabilities,...SuggestedFull timeRemote workWaymo
San Francisco, CA1 day ago$170k - $216k
...billions in simulation across 15+ U.S. states. The Planner Evaluation team works on one of the key challenges in autonomous driving... ...C++ Experience with ML Experience with A/B experiment infrastructure Experience building and validating metrics to measure...SuggestedFull timeRemote workWaymo
San Francisco, CA1 day ago$166k - $220k
...This position is on the TRS Core Infrastructure team and will be focused on building the... ...streamlined user interfaces. Test and evaluate your software at multiple levels of rigor... ...systems, and iterating upon designs via a technical design review process. Familiarity...Full timeWork experience placementImmediate startRelocation packageAnduril Industries
Costa Mesa, CA1 day ago$200k - $300k
...Skild AI, Inc. seeks a Senior Software Engineer, AI Training & Infrastructure in San Mateo, CA. You will be responsible for building and... ...full ML lifecycle—data preparation, training orchestration, evaluation, and deployment—for real-world robotics applications. This includes...Full timeSkild AI
San Mateo, CA1 day ago$169.9k - $207.6k
...needs.We are seeking a highly experienced Sr. Manager, IT Infrastructure Technical Lead responsible for driving architecture, delivery,... ...all Envista Companies are equal opportunity employers that evaluate qualified applicants without regard to race, color, national...Full timeContract workWork at officeRemote workNight shift$160.36k - $240.54k
...one of the following areas: Data Platform, Simulation, and Technical Infrastructure. Data Platform: The Data Platform serves as a... ...search. Additionally, the platform supports the autonomy evaluation infrastructure by providing detailed introspection. Simulation...Full timeNuro
Mountain View, CA1 day ago$180k
...software engineers to build the software infrastructure that enables our models to use tools... ...reinforcement learning, agentic AI, and model evaluation. Focus Develop infrastructure for... ...main process, which consists of four technical interviews: # Coding assessment in a...Full timeLocal areaRelocationXai
Palo Alto, CA1 day ago$115k - $190k
...motivated Software Engineer, Developer Infrastructure to design, build, and maintain the... ...engineering, product, field deployment, and technical communication, all focused on shipping... ...fostering an inclusive workplace. We evaluate candidates and employees based on merit...Full timeLocal areaFieldai
Irvine, CA1 day ago$145.4k - $188.1k
...together. About the Machine Learning Infrastructure Team At Thumbtack, our challenges... ...CI/CD, orchestration, deployment, and evaluation tooling. Work hands on across the stack... ...measurable outcomes. Solve complex technical problems and stay up to date with advances...Full timeSeasonal workLocal areaThumbtack
San Francisco, CA1 day ago$130k - $175k
...practices and help guide new and existing services. Practicing Infrastructure as Code (IaC) wherever possible, giving us the confidence in... ...resiliency Support the engineering team with tools to evaluate the performance of their code in production environments, speed...Permanent employmentFull timeWork at officeLocal areaRemote workKiddom
San Francisco, CA1 day ago$131k - $176k
...Wanna join the adventure? As a Space Infrastructure Software Engineer, you are responsible for scaling our ability to operate a heterogeneous... .... The salary range for this role is intentionally wide as we evaluate individuals based on their unique experience and abilities to...Full timeTemporary workWork at officeRelocation packageFlexible hoursLoft Orbital Solutions
San Francisco, CA1 day ago- ...Forces. Role + Responsibilities Specter is hiring an ML infrastructure engineer to build and scale the machine learning systems that... ..., ONNX, CoreML). Developing continuous training and evaluation systems to improve model performance from production data feedback...Full time
Specter
San Francisco, CA1 day ago $160.36k - $240.54k
...with internal stakeholders and external suppliers to define, evaluate, and integrate the next-generation HW platform for Nuro’s products... ...tuning and optimization;profiling and tracing tools and infrastructure (perf, eBPF, Perfetto, pprof, NVIDIA Nsight Systems/Compute);...Full timeNuro
Mountain View, CA1 day ago- ...Workspace. What you'll do The Payments Infrastructure team builds the trust boundary between... ...engineer in the loop. Set technical direction across the team's hardest problems... ...the job description. We strive to evaluate all applicants consistently without regard...Full timeFlexible hours
Sierra Limited
San Francisco, CA1 day ago $140k - $225k
...Software Engineer, Tooling and Development Infrastructure, you will play a critical role in... ...systems to user-facing features. Identify, evaluate, and integrate new tools that streamline... ...-as-Code (IaC) solutions. Lead continuous improvement efforts for our build...Full timeTemporary workLocal areaFlexible hoursHP IQ
San Francisco, CA1 day ago$152k - $228k
...leverages many different bench-top systems to evaluate and regression test different aspects... ...it reaches the road. You will own the infrastructure that makes this possible. Our... ...the entire autonomy stack. You'll be the technical DRI for the platform — setting the roadmap...Full timeTemporary workWho We Are Nuro
Mountain View, CA1 day ago$135k - $170k
...looking for a Software Engineer, Test Infrastructure to build and scale the systems that ensure... ..., product, field deployment, and technical communication, all focused on shipping... ...to fostering an inclusive workplace. We evaluate candidates and employees based on merit...Full timeLocal areaFieldai
Irvine, CA1 day ago$215k - $265k
...and maintain Apollo's cloud infrastructure . This means IaC, networking... ...Frontier labs trust us to evaluate their models pre-release. It... ...work closely with our security lead to make sure the... ...test (approx. 2 hours) ~3 technical interviews ~ Final interview...Full timeWork experience placementWork at officeImmediate startVisa sponsorshipFlexible hoursApollo Research
San Francisco, CA1 day ago- ...Anrok is the leading tax automation platform enabling businesses... ...We're building the agentic infrastructure layer at Anrok—the systems that... ...autonomous systems. Own technical architecture decisions for... ...and iteration. Build the evaluation, observability, and reliability...Full timeWork at officeRemote workHome officeFlexible hours3 days per week
Anrok, Inc.
San Francisco, CA1 day ago - ...Summary We are looking for an experienced Growth Infrastructure Engineer to build and maintain the technical backbone that enables scalable growth experiments... ...systems. Enable reliable data collection and evaluation for growth experiments. Automate deployment...Full timeTemporary workWork at officeWorldwideMonday to FridayFlexible hours
Replit
Foster, CA1 day ago $170k - $216k
...across 15+ U.S. states. The Simulation Infrastructure team creates reliable, scalable, and cost... ...effective Simulation-based products that evaluate the Waymo Driver's software stack at a massive scale. We solve complex technical challenges to build services and tools...Full timeRemote workWaymo
San Francisco, CA1 day ago$204k - $259k
...nominal performance Develop necessary high scale performance evaluation, debugging and software change management processes... ...using profile guided techniques We prefer: 2 years in infrastructure/systems/performance domain optimizing end-to-end systems 2...Full timeWork at officeRemote workWaymo
Mountain View, CA1 day ago- ...Software Engineer, Agent Evaluation and Quality Engineering · Full-time · San Francisco... ..., evaluation, and feedback-loop infrastructure that makes the Cursor core agent reliably... ...fit, we'll reach to schedule 2-3 short technicals. After, we'll schedule an onsite in...Full timeWork at office
Anysphere
San Francisco, CA1 day ago - ...About the Role The Infrastructure Engineering team is crucial... ...Architect the Service Backbone: Lead the design and evolution of... ...engineering teams. Drive Technical Roadmap: Define the long-term... ...deep-dive code reviews, and evaluate technical trade-offs to...Shift work
Hayden AI
San Francisco, CA5 days ago - ...Senior Software Engineer (ML Infrastructure) Arena Intelligence is seeking a Senior Software Engineer (ML Infrastructure) to lead the design and development of scalable, high-performance... ...trustworthy, transparent, and timely evaluation signals across the platform. This role...Permanent employment
Arena AI
San Francisco, CA12 hours ago - ...engineered for the management of both infrastructure and applications. You will have the opportunity... ...will actively participate in guided technical labs covering prompt engineering,... ...orchestration, fine-tuning strategies, and evaluation techniques. You will develop, refine,...H1bImmediate startWork visa
Spectro Cloud
San Jose, CA1 day ago - ...As a Software Engineer, Infrastructure at Sierra, you will be responsible... ...health and performance. Lead and participate in incident... ...development experience in highly technical products. A strong... ...job description. We strive to evaluate all applicants consistently...Full timeFlexible hours
Sierra
San Francisco, CA3 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Technical Lead, Evaluation Infrastructure. Be the first to apply!
