Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

(Senior) Software Engineer - Evaluation

$120k - $250k
Full-time

Pony Ai

Description

Founded in 2016 in Silicon Valley, Pony.ai has quickly become a global leader in autonomous mobility and is a pioneer in extending autonomous mobility technologies and services at a rapidly expanding footprint of sites around the world. Operating Robotaxi, Robotruck and Personally Owned Vehicles (POV) business units, Pony.ai is an industry leader in the commercialization of autonomous driving and is committed to developing the safest autonomous driving capabilities on a global scale. Pony.ai’s leading position has been recognized, with CNBC ranking Pony.ai #10 on its CNBC Disruptor list of the 50 most innovative and disruptive tech companies of 2022. In June 2023, Pony.ai was recognized on the XPRIZE and Bessemer Venture Partners inaugural “XB100” 2023 list of the world’s top 100 private deep tech companies, ranking #12 globally. As of August 2023, Pony.ai has accumulated nearly 21 million miles of autonomous driving globally. Pony.ai went public at NASDAQ in November 2024.

Responsibility

  • Design and build end-to-end data processing pipelines and toolchains to support large-scale data collection, batch processing, curation, and multi-dimensional evaluation.
  • Design and implement high-performance backend services and distributed architectures , capable of handling large-scale real-time event processing.
  • Collaborate across complex business components to lead the design of targeted system-level evaluation metrics, testing frameworks, and advanced simulation environments.
  • Leverage and improve industry-leading, open-source distributed technologies to continuously advance infrastructure capabilities and processing efficiency.
  • Drive critical architectural decisions and technology selection, ensuring rigorous standards for security, stability, scalability, and high performance.
  • Establish and maintain comprehensive system monitoring, fine-grained observability metrics, and automated alerting frameworks to ensure low latency and reliable production operations.
  • Build and optimize downstream engineering workflows for Large Language Models (LLMs), programmatically steering model reasoning, establishing guardrails, and mitigating hallucinations to ensure reliable outputs.

Requirements

  • Strong programming skills in C/C++, Python, and software design
  • Strong foundation in mathematics and theoretical computer science, such as linear algebra, calculus, graph theory, computational geometry, combinatorial optimization algorithms, stochastic processes, and complexity analysis.
  • Possessing solid engineering discipline, or demonstrating strong interest and potential in building large-scale systems, with the ability to maximize performance while keeping complexity and cost to a minimum.
  • Pursueing a BS/MS or Ph.D in Computer Science or a related field
  • Experience in large data set processing and familiarity with real time systems
  • Solid experience in a fast-paced and structured engineering environment
  • Full stack experience including both front end and back end is preferred
  • Statistics analysis experience is preferred

Compensation and Benefits

Base Salary Range: $120,000 - $250,000 Annually

Compensation may vary outside of this range depending on many factors, including the candidate’s qualifications, skills, competencies, experience, and location. Base pay is one part of the Total Compensation and this role may be eligible for bonuses/incentives and restricted stock units.

Also, we provide the following benefits to eligible employees:

  • Health Care Plan (Medical, Dental & Vision)
  • Retirement Plan (Traditional and Roth 401k)
  • Life Insurance (Basic, Voluntary & AD&D)
  • Paid Time Off (Vacation & Public Holidays)
  • Family Leave (Maternity, Paternity)
  • Short Term & Long Term Disability
  • Free Food & Snacks

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the (Senior) Software Engineer - Evaluation in Remote vacancy
  • $224k - $356.5k

     ...the future of autonomous driving, and evaluation is how we know the drive is getting better...  ...AI drivers!We are looking for a senior engineer to own the engine that makes it all trustworthy...  ...experience).12+ years building software, with significant time in autonomous vehicles... 
    Senior
    Full time
    Remote work

    Nvidia

    Santa Clara, CA
    1 day ago
  • $184k - $287.5k

     ...Autonomous Vehicles.We are looking for an experienced engineer to join our Planning and Control team to work on metrics and evaluation. In this role you will enable rapid planner...  ..., and evaluation of our Autonomous Vehicle software.Build compelling, data driven evaluation... 
    Senior
    Full time
    Remote work

    Nvidia

    Santa Clara, CA
    5 days ago
  • $129.4k - $198.4k

     ...experiences. About the Organization:The Evaluation team builds and evolves the evaluation...  ...simulation tests used for autonomous vehicle software validation. Develop and maintain GM’s...  ..., and interpretable insights to engineering teams and leadership, including trend analysis... 
    Senior
    Full time
    Local area
    Remote work
    Work from home
    Flexible hours

    General Motors

    Sunnyvale, TX
    4 days ago
  • $144.7k - $221.4k

     ...experiences. About the Organization The Evaluation team builds and evolves the evaluation...  ...-level results into clear feedback for engineering and leadership, and help accelerate validated...  ...to introspect autonomous driving software performance at interfaces across the autonomy... 
    Senior
    Full time
    Local area
    Remote work
    Work from home
    Relocation
    Relocation package
    Flexible hours

    General Motors

    Sunnyvale, TX
    3 days ago
  • $204k - $259k

     ...Senior Software Engineer, Quantitative Evaluations Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver... 
    Senior
    Full time
    Remote work

    Waymo

    San Francisco, CA
    1 day ago
  • $80 - $100 per hour

     ...Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: Design coding benchmarks that evaluate...  ...their country of residence. Nice to have Senior or Lead-level profile with a history of... 
    Senior
    Remote job
    Full time
    Contract work
    For contractors

    G2i

    Miami, FL
    1 day ago
  • $204k - $259k

     ...other Waymo teams that consume map data. You will: Evaluate and launch machine learning models that automatically generate...  ...work on various parts of the system ~ A passion for good software engineering We prefer: M.S. or Ph.D in Computer Science or a similar... 
    Senior
    Full time
    Remote work

    Waymo

    Remote
    1 day ago
  •  ...Role Overview Build and validate verifiable software engineering tasks and evaluation datasets derived from public repository histories, to measure and...  ...the project. Qualifications Proven experience at a senior or tech-lead level working with high-quality public... 
    Senior
    For contractors
    Local area
    Remote work

    SaidGig

    United States
    more than 2 months ago
  •  ...Role Overview Design and evaluate high-quality datasets and evaluations that advance large language models for code. You will work...  ...detect errors and measure model capabilities across the software engineering lifecycle. Key Responsibilities Curate and author code... 
    Senior
    For contractors
    Remote work
    10 hours per week
    Flexible hours

    SaidGig

    Canada
    more than 2 months ago
  •  ...experiences. About the Organization: The Evaluation team builds and evolves the evaluation...  ...provide clear, explainable insights to engineering and leadership, including trend...  ...design, code reviews, and by following software-engineering best practices. Interface with... 
    Senior
    Local area
    Work from home

    General Motors

    Sunnyvale, CA
    5 days ago
  •  ...Role Overview Drive development of high-quality datasets and evaluation pipelines that improve and benchmark large language models for code generation and software engineering tasks. You will curate and author reference code, evaluate and refine AI-generated solutions... 
    Senior
    Full time
    For contractors
    Remote work
    10 hours per week
    Flexible hours

    SaidGig

    United States
    more than 2 months ago
  • $40 per hour

     ...leading tech companies, focused on testing, evaluating, and improving AI systems....  ...NOT: Not data labeling. Not prompt engineering. Not writing code from scratch - the...  ...evaluate. Qualifications ~5+ years in software development. ~Core stack: Python (FastAPI... 
    Senior
    Part time

    Mindrift

    Remote
    4 days ago
  • $180k - $240k

    Role Description Deepgram is looking for a Senior Software Engineer - Model Evaluation & AI Systems to join the team responsible for validating the quality of our speech, audio, and multilingual models before they reach customers. This team owns the evaluation and quality... 
    Senior
    Full time

    Deepgram

    Remote
    3 days ago
  • $125k - $175k

     ...NinjaTrader equips traders with award-winning software and brokerage services to navigate...  ...'ll do:We are seeking a Sr. Software Engineer to join our Evaluation Services team. Our Evaluation...  ...same.In this role you will:Serve as the senior technical point of contact for... 
    Senior
    Work at office
    Remote work
    Worldwide
    Monday to Friday
    Flexible hours
    Shift work

    NinjaTrader Group

    Chicago, IL
    3 days ago
  • $127k - $223k

     ...positive way. To learn more visit: The Evaluation Algorithms team is responsible for...  ...highly realistic closed-loop simulation engine built with the latest in generative AI technologies...  ...in Python programming and strong software engineering fundamentals with real-world... 
    Senior
    Full time
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    28 days ago
  • $40 per hour

     ...Role Overview Lead hands-on engineering work to create and validate verifiable software engineering tasks drawn from public repository histories, and evaluate how large language models perform on...  ...software engineering experience at a senior or tech lead level, with... 
    Senior
    Contract work
    For contractors
    Local area
    Remote work

    SaidGig

    United States
    2 days ago
  • $50 per hour

     ...Role Overview Help build and evaluate the datasets and tooling that advance large language models for real-world software engineering. You will collaborate with researchers to curate code examples, author precise solutions, and correct AI-generated code across multiple... 
    Senior
    Full time
    For contractors
    Remote work
    10 hours per week
    Flexible hours

    SaidGig

    United States
    2 days ago
  • $160k - $180k

     ...Greenfield Engineering, a Saalex Company, is seeking a  Senior Software Engineer in Patuxent River, MD . This role involves applying advanced engineering principles to design, develop, and evaluate mission-critical military and aviation systems, ensuring compliance... 
    Senior
    Full time

    Saalex

    Remote
    1 day ago
  • $85k - $100k

     ...Full-time Description Job Description: Software Engineer Position Overview The Software Engineer will provide critical engineering support in the design, development, testing, evaluation, and sustainment of software systems for Air Force and joint service... 
    Senior
    Full time

    Diaconia

    Remote
    1 day ago
  • $75 per hour

    Role Description We are hiring senior full-stack engineers to help build and refine next-generation coding environments that evaluate AI agents. This work directly supports the development...  ...~Senior-level experience in full-stack software engineering ~Deep professional... 
    Senior
    Hourly pay
    Contract work
    Remote work

    Terac

    Remote
    5 days ago
  •  ...over sponsorship of an employment visa at this time. Senior Software Engineer   In this role, you will: Lead moderately complex initiatives...  ...complex technical challenges that require an in-depth evaluation of technologies and procedures Resolve moderately... 
    Senior
    Work experience placement
    Remote work
    Work visa

    INFOSYS NOVA HOLDINGS

    Phoenix, AZ
    19 days ago
  •  ...billion dollar HR tech industry. Job Description The Senior Software Engineer works as a leader/key player on our development team to...  ...Definition of requirements and design specifications # Evaluation of system integration requirements with other providers of... 
    Senior
    Full time

    Chequed.com

    Remote
    1 day ago
  • $140k - $175k

     ...development of the Gentuity imaging application software and internal software tools ·...  ...features, GUI ideas, and for hardware evaluation. · Participate in evaluation and selection...  ...of 5 years of experience in Software Engineering · Modern C++, including Objected... 
    Senior
    Full time

    Gentuity, LLC

    Remote
    1 day ago
  • $161.89k - $193.7k

     ...procedures for new assignments and select approaches through the evaluation and adaptation of complex techniques. Contribute to the...  ...Implement REST services, XML, and XSD using MVC architecture. Software development and implementation using JavaScript, C#, jQuery... 
    Senior
    Full time

    Convera

    Remote
    1 day ago
  •  ...LiDAR, Radar) Build and maintain data collection, labeling and evaluation (metrics) pipeline Maintain proprietary multi-modal dataset...  ...a deterministic secondary perception system Mentor junior engineers about best practices You have ~ Bachelor’s or Master’s... 
    Senior
    Remote job
    Full time

    Aerovect

    Remote
    1 day ago
  •  ...Overview \ \ We are seeking an experienced Software Engineer to develop, maintain, and enhance complex software systems supporting mission...  ..., including hardware/software trade -off analysis. \\ Evaluate opportunities for software reuse and integration of Commercial... 
    Senior
    Full time
    Local area

    J

    Remote
    1 day ago
  •  ...perception Build and maintain radar data collection, labeling, and evaluation pipelines Architect a deterministic secondary perception system with radar as a primary modality Mentor junior engineers on radar perception best practices You have ~ Bachelor's or... 
    Senior
    Full time

    Aerovect

    Remote
    1 day ago
  •  ...well-structured design Partner closely with leadership and engineering to translate product goals into pragmatic solutions Help...  ...infrastructure, developer tooling, agent frameworks, or replayable evaluation environments (e.g. Strandloop) ~ Strong communication and... 
    Senior
    Remote work

    ProNexus LLC

    New York, NY
    13 days ago
  •  ...smarter and more capable. The Role We’re looking for senior software engineers to help us realize TruckSmarter’s vision of eliminating...  ...Stay ahead of emerging AI frameworks and technologies, evaluating and introducing them when they create real product advantage... 
    Senior
    Full time
    Work at office
    Flexible hours

    Trucksmarter, Inc.

    Remote
    1 day ago
  • $95k - $150k

     ...only the best from us. RSC2 is currently looking for a Senior Software Engineer to join our growing team in Aberdeen to support our SAI...  ...Salary-Exempt  We are an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion... 
    Senior
    Full time
    Contract work
    Part time

    Rsc2, Inc.

    Remote
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to (Senior) Software Engineer - Evaluation. Be the first to apply!