Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

(Senior) Software Engineer - Evaluation

$120k - $250k
Full-time

Pony Ai

Description

Founded in 2016 in Silicon Valley, Pony.ai has quickly become a global leader in autonomous mobility and is a pioneer in extending autonomous mobility technologies and services at a rapidly expanding footprint of sites around the world. Operating Robotaxi, Robotruck and Personally Owned Vehicles (POV) business units, Pony.ai is an industry leader in the commercialization of autonomous driving and is committed to developing the safest autonomous driving capabilities on a global scale. Pony.ai’s leading position has been recognized, with CNBC ranking Pony.ai #10 on its CNBC Disruptor list of the 50 most innovative and disruptive tech companies of 2022. In June 2023, Pony.ai was recognized on the XPRIZE and Bessemer Venture Partners inaugural “XB100” 2023 list of the world’s top 100 private deep tech companies, ranking #12 globally. As of August 2023, Pony.ai has accumulated nearly 21 million miles of autonomous driving globally. Pony.ai went public at NASDAQ in November 2024.

Responsibility

  • Design and build end-to-end data processing pipelines and toolchains to support large-scale data collection, batch processing, curation, and multi-dimensional evaluation.
  • Design and implement high-performance backend services and distributed architectures , capable of handling large-scale real-time event processing.
  • Collaborate across complex business components to lead the design of targeted system-level evaluation metrics, testing frameworks, and advanced simulation environments.
  • Leverage and improve industry-leading, open-source distributed technologies to continuously advance infrastructure capabilities and processing efficiency.
  • Drive critical architectural decisions and technology selection, ensuring rigorous standards for security, stability, scalability, and high performance.
  • Establish and maintain comprehensive system monitoring, fine-grained observability metrics, and automated alerting frameworks to ensure low latency and reliable production operations.
  • Build and optimize downstream engineering workflows for Large Language Models (LLMs), programmatically steering model reasoning, establishing guardrails, and mitigating hallucinations to ensure reliable outputs.

Requirements

  • Strong programming skills in C/C++, Python, and software design
  • Strong foundation in mathematics and theoretical computer science, such as linear algebra, calculus, graph theory, computational geometry, combinatorial optimization algorithms, stochastic processes, and complexity analysis.
  • Possessing solid engineering discipline, or demonstrating strong interest and potential in building large-scale systems, with the ability to maximize performance while keeping complexity and cost to a minimum.
  • Pursueing a BS/MS or Ph.D in Computer Science or a related field
  • Experience in large data set processing and familiarity with real time systems
  • Solid experience in a fast-paced and structured engineering environment
  • Full stack experience including both front end and back end is preferred
  • Statistics analysis experience is preferred

Compensation and Benefits

Base Salary Range: $120,000 - $250,000 Annually

Compensation may vary outside of this range depending on many factors, including the candidate’s qualifications, skills, competencies, experience, and location. Base pay is one part of the Total Compensation and this role may be eligible for bonuses/incentives and restricted stock units.

Also, we provide the following benefits to eligible employees:

  • Health Care Plan (Medical, Dental & Vision)
  • Retirement Plan (Traditional and Roth 401k)
  • Life Insurance (Basic, Voluntary & AD&D)
  • Paid Time Off (Vacation & Public Holidays)
  • Family Leave (Maternity, Paternity)
  • Short Term & Long Term Disability
  • Free Food & Snacks

Vacancy posted 12 hours ago
Similar jobs that could be interesting for youBased on the (Senior) Software Engineer - Evaluation in Remote vacancy
  • $204k - $259k

     ...create a training ground for the Waymo Driver. The Simulator Evaluation team faces the ultimate data challenge: How do you...  ...prove that a virtual world is "real"? We are looking for a Senior Software Engineer to build the metrics and systems that grade this hybrid environment... 
    Senior
    Full time
    Remote work

    Waymo

    San Francisco, CA
    12 hours ago
  • $224k - $356.5k

     ...the future of autonomous driving, and evaluation is how we know the drive is getting better...  ...AI drivers!We are looking for a senior engineer to own the engine that makes it all trustworthy...  ...experience).12+ years building software, with significant time in autonomous vehicles... 
    Senior
    Full time
    Remote work

    Nvidia

    Santa Clara, CA
    2 days ago
  •  ...experiences. About the Organization The Evaluation team builds and evolves the evaluation...  ...-level results into clear feedback for engineering and leadership, and help accelerate validated...  ...to introspect autonomous driving software performance at subsystem interfaces across... 
    Senior
    Full time
    Local area
    Work from home

    General Motors

    Sunnyvale, TX
    1 day ago
  • $238k - $302k

     ...that are core to our autonomous driving software. We help our partners by offering the best...  ...lifecycle, including training and evaluation. They are geared towards both scaling models...  ...autonomous driving.  We are looking for engineers with ML system expertise to help us... 
    Senior
    Full time
    Remote work

    Waymo

    Remote
    12 hours ago
  • $204k - $259k

     ...Waymo Planner Evaluation Team Role Waymo is an autonomous driving technology company with...  ...measuring and improving the quality of the software that drives the car. We are looking for experienced data-minded software engineers and data scientists to help us improve... 
    Senior
    Full time
    Remote work

    Latent Logic

    Mountain View, CA
    3 days ago
  • $204k - $259k

     ...billions in simulation across 15+ U.S. states. Waymo's Release Evaluation org ensures that each version of the Waymo Driver is safe...  ...objectives under resource constraints. Collaborate with other engineers, data scientists, statisticians and the leadership team to... 
    Senior
    Full time
    Remote work

    Waymo

    Mountain View, CA
    1 day ago
  • $80 - $100 per hour

     ...Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: Design coding benchmarks that evaluate...  ...their country of residence. Nice to have Senior or Lead-level profile with a history of... 
    Senior
    Remote job
    Full time
    Contract work
    For contractors

    G2i

    Miami, FL
    12 hours ago
  •  ...and scale infrastructure for inference, evaluation, and continuous model improvement...  ...with a strong blend of infrastructure engineering, production ML systems, LLM/LVM inference...  ...background, especially in Python, with solid software engineering fundamentals. ~... 
    Senior
    Full time
    Work at office
    Local area
    Flexible hours
    3 days per week

    Ambient.ai

    Remote
    12 hours ago
  •  ...Role Overview Build and validate verifiable software engineering tasks and evaluation datasets derived from public repository histories, to measure and...  ...the project. Qualifications Proven experience at a senior or tech-lead level working with high-quality public... 
    Senior
    For contractors
    Local area
    Remote work

    SaidGig

    United States
    more than 2 months ago
  •  ...Role Overview Design and evaluate high-quality datasets and evaluations that advance large language models for code. You will work...  ...detect errors and measure model capabilities across the software engineering lifecycle. Key Responsibilities Curate and author code... 
    Senior
    For contractors
    Remote work
    10 hours per week
    Flexible hours

    SaidGig

    Canada
    more than 2 months ago
  •  ...Role Overview Drive development of high-quality datasets and evaluation pipelines that improve and benchmark large language models for code generation and software engineering tasks. You will curate and author reference code, evaluate and refine AI-generated solutions... 
    Senior
    Full time
    For contractors
    Remote work
    10 hours per week
    Flexible hours

    SaidGig

    United States
    more than 2 months ago
  • Role Description We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. Essentially...  ...scope and deeper involvement. Qualifications ~Strong software engineering experience: 8+ years in any modern programming language,... 
    Senior
    Temporary work
    Freelance
    Remote work
    Flexible hours

    Toloka

    Remote
    20 days ago
  •  ...The Opportunity Deepgram is looking for a Senior Software Engineer - Model Evaluation & AI Systems to join the team responsible for validating the quality of our speech, audio, and multilingual models before they reach customers. This team owns the evaluation and... 
    Senior
    Full time

    Deepgram

    Remote
    a month ago
  • $125k - $175k

     ...NinjaTrader equips traders with award-winning software and brokerage services to navigate...  ...'ll do:We are seeking a Sr. Software Engineer to join our Evaluation Services team. Our Evaluation...  ...same.In this role you will:Serve as the senior technical point of contact for... 
    Senior
    Work at office
    Remote work
    Worldwide
    Monday to Friday
    Flexible hours
    Shift work

    NinjaTrader Group

    Chicago, IL
    3 days ago
  • $125k - $175k

     ...starting out, NinjaTrader equips traders with award-winning software and brokerage services to navigate the world's leading...  ...world. What you'll do: We are seeking a Sr. Software Engineer to join our Evaluation Services team. Our Evaluation Services business powers funded... 
    Senior
    Full time
    Work at office
    Remote work
    Worldwide
    Monday to Friday
    Flexible hours
    Shift work

    NinjaTrader

    Chicago, IL
    2 days ago
  • $40 per hour

     ...Role Overview Lead hands-on engineering work to create and validate verifiable software engineering tasks drawn from public repository histories, and evaluate how large language models perform on...  ...software engineering experience at a senior or tech lead level, with... 
    Senior
    Contract work
    For contractors
    Local area
    Remote work

    SaidGig

    United States
    5 days ago
  • $50 per hour

     ...Role Overview Help build and evaluate the datasets and tooling that advance large language models for real-world software engineering. You will collaborate with researchers to curate code examples, author precise solutions, and correct AI-generated code across multiple... 
    Senior
    Full time
    For contractors
    Remote work
    10 hours per week
    Flexible hours

    SaidGig

    United States
    2 days ago
  • $85k - $100k

     ...Full-time Description Job Description: Software Engineer Position Overview The Software Engineer will provide critical engineering support in the design, development, testing, evaluation, and sustainment of software systems for Air Force and joint service... 
    Senior
    Full time

    Diaconia

    Remote
    12 hours ago
  •  ...billion dollar HR tech industry. Job Description The Senior Software Engineer works as a leader/key player on our development team to...  ...Definition of requirements and design specifications # Evaluation of system integration requirements with other providers of... 
    Senior
    Full time

    Chequed.com

    Remote
    12 hours ago
  • $140k - $175k

     ...development of the Gentuity imaging application software and internal software tools ·...  ...features, GUI ideas, and for hardware evaluation. · Participate in evaluation and selection...  ...of 5 years of experience in Software Engineering · Modern C++, including Objected... 
    Senior
    Full time

    Gentuity, LLC

    Remote
    12 hours ago
  •  ...LiDAR, Radar) Build and maintain data collection, labeling and evaluation (metrics) pipeline Maintain proprietary multi-modal dataset...  ...a deterministic secondary perception system Mentor junior engineers about best practices You have ~ Bachelor’s or Master’s... 
    Senior
    Remote job
    Full time

    Aerovect

    Remote
    12 hours ago
  •  ...Overview \ \ We are seeking an experienced Software Engineer to develop, maintain, and enhance complex software systems supporting mission...  ..., including hardware/software trade -off analysis. \\ Evaluate opportunities for software reuse and integration of Commercial... 
    Senior
    Full time
    Local area

    J

    Remote
    12 hours ago
  •  ...perception Build and maintain radar data collection, labeling, and evaluation pipelines Architect a deterministic secondary perception system with radar as a primary modality Mentor junior engineers on radar perception best practices You have ~ Bachelor's or... 
    Senior
    Full time

    Aerovect

    Remote
    12 hours ago
  •  ...Voltai is developing models and agents to evaluate, design, and interact with hardware and...  ...keyword-match ~5+ years in backend engineering, full-stack development, or infrastructure...  ...EDA, semiconductor, or hardware-adjacent software environments Our AI Bullshit Meter... 
    Senior
    Full time
    Internship
    Work at office

    Voltai

    Remote
    12 hours ago
  • $95k - $150k

     ...only the best from us. RSC2 is currently looking for a Senior Software Engineer to join our growing team in Aberdeen to support our SAI...  ...Salary-Exempt  We are an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion... 
    Senior
    Full time
    Contract work
    Part time

    Rsc2, Inc.

    Remote
    12 hours ago
  • $160k - $210k

     ...businesses. We're seeking an experienced Engineer to develop and build innovative data...  ...interactions with our data. Engage with software engineers, data scientists, and product...  ...that move a customer from discovery to evaluation to purchase. Analyze and extract value... 
    Senior
    Full time
    Temporary work

    Enigma

    Remote
    12 hours ago
  • $213k - $270k

     ...identify, and guide fixes of reliability and software integrity issues.  We focus the...  ...role you will report to a Staff Software Engineer / Tech Lead Manager.   You will:...  ...-scale cloud issues and their impact on evaluation vs. onroad behavior Automate measurement... 
    Senior
    Full time
    Remote work

    Waymo

    Mountain View, CA
    12 hours ago
  •  ...observability platform built on the Snowflake AI Data Cloud and engineered for scale. We ingest and store logs, metrics, traces, and...  ...scale. Experience with query DSLs, CEL, or similar expression evaluation frameworks. Prior experience in a tech lead or staff engineer... 
    Senior
    Full time

    Snowflake

    Remote
    12 hours ago
  • $50 - $70 per hour

     ...Location: Remote Role Responsibilities Develop and complete coding tasks in various enterprise languages . Evaluate and improve systems and build toolchains using C and Rust . Enhance enterprise JVM and build ecosystems with C#... 
    Senior
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    Remote
    12 hours ago
  •  ...Department: IT Development Reports to: Software Engineering Manager Location: Hybrid - Dublin, Ohio (3 days) Remote (2 days) Full-...  ...current with emerging technologies and industry trends. Evaluate frameworks, technology, platforms, and tooling solutions.... 
    Senior
    Full time
    Remote work

    T-Cetra

    Dublin, OH
    12 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to (Senior) Software Engineer - Evaluation. Be the first to apply!