(Senior) Software Engineer - Evaluation
$120k - $250kPony Ai
Founded in 2016 in Silicon Valley, Pony.ai has quickly become a global leader in autonomous mobility and is a pioneer in extending autonomous mobility technologies and services at a rapidly expanding footprint of sites around the world. Operating Robotaxi, Robotruck and Personally Owned Vehicles (POV) business units, Pony.ai is an industry leader in the commercialization of autonomous driving and is committed to developing the safest autonomous driving capabilities on a global scale. Pony.ai’s leading position has been recognized, with CNBC ranking Pony.ai #10 on its CNBC Disruptor list of the 50 most innovative and disruptive tech companies of 2022. In June 2023, Pony.ai was recognized on the XPRIZE and Bessemer Venture Partners inaugural “XB100” 2023 list of the world’s top 100 private deep tech companies, ranking #12 globally. As of August 2023, Pony.ai has accumulated nearly 21 million miles of autonomous driving globally. Pony.ai went public at NASDAQ in November 2024.
Responsibility
- Design and build end-to-end data processing pipelines and toolchains to support large-scale data collection, batch processing, curation, and multi-dimensional evaluation.
- Design and implement high-performance backend services and distributed architectures , capable of handling large-scale real-time event processing.
- Collaborate across complex business components to lead the design of targeted system-level evaluation metrics, testing frameworks, and advanced simulation environments.
- Leverage and improve industry-leading, open-source distributed technologies to continuously advance infrastructure capabilities and processing efficiency.
- Drive critical architectural decisions and technology selection, ensuring rigorous standards for security, stability, scalability, and high performance.
- Establish and maintain comprehensive system monitoring, fine-grained observability metrics, and automated alerting frameworks to ensure low latency and reliable production operations.
- Build and optimize downstream engineering workflows for Large Language Models (LLMs), programmatically steering model reasoning, establishing guardrails, and mitigating hallucinations to ensure reliable outputs.
- Strong programming skills in C/C++, Python, and software design
- Strong foundation in mathematics and theoretical computer science, such as linear algebra, calculus, graph theory, computational geometry, combinatorial optimization algorithms, stochastic processes, and complexity analysis.
- Possessing solid engineering discipline, or demonstrating strong interest and potential in building large-scale systems, with the ability to maximize performance while keeping complexity and cost to a minimum.
- Pursueing a BS/MS or Ph.D in Computer Science or a related field
- Experience in large data set processing and familiarity with real time systems
- Solid experience in a fast-paced and structured engineering environment
- Full stack experience including both front end and back end is preferred
- Statistics analysis experience is preferred
Compensation and Benefits
Base Salary Range: $120,000 - $250,000 Annually
Compensation may vary outside of this range depending on many factors, including the candidate’s qualifications, skills, competencies, experience, and location. Base pay is one part of the Total Compensation and this role may be eligible for bonuses/incentives and restricted stock units.
Also, we provide the following benefits to eligible employees:
- Health Care Plan (Medical, Dental & Vision)
- Retirement Plan (Traditional and Roth 401k)
- Life Insurance (Basic, Voluntary & AD&D)
- Paid Time Off (Vacation & Public Holidays)
- Family Leave (Maternity, Paternity)
- Short Term & Long Term Disability
- Free Food & Snacks
$224k - $356.5k
...the future of autonomous driving, and evaluation is how we know the drive is getting better... ...AI drivers!We are looking for a senior engineer to own the engine that makes it all trustworthy... ...experience).12+ years building software, with significant time in autonomous vehicles...SeniorFull timeRemote work$184k - $287.5k
...Autonomous Vehicles.We are looking for an experienced engineer to join our Planning and Control team to work on metrics and evaluation. In this role you will enable rapid planner... ..., and evaluation of our Autonomous Vehicle software.Build compelling, data driven evaluation...SeniorFull timeRemote work$129.4k - $198.4k
...experiences. About the Organization:The Evaluation team builds and evolves the evaluation... ...simulation tests used for autonomous vehicle software validation. Develop and maintain GM’s... ..., and interpretable insights to engineering teams and leadership, including trend analysis...SeniorFull timeLocal areaRemote workWork from homeFlexible hours$144.7k - $221.4k
...experiences. About the Organization The Evaluation team builds and evolves the evaluation... ...-level results into clear feedback for engineering and leadership, and help accelerate validated... ...to introspect autonomous driving software performance at interfaces across the autonomy...SeniorFull timeLocal areaRemote workWork from homeRelocationRelocation packageFlexible hours$204k - $259k
...Senior Software Engineer, Quantitative Evaluations Waymo is an autonomous driving technology company with the mission to be the world's most trusted driver. Since its start as the Google Self-Driving Car Project in 2009, Waymo has focused on building the Waymo Driver...SeniorFull timeRemote work$80 - $100 per hour
...Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: Design coding benchmarks that evaluate... ...their country of residence. Nice to have Senior or Lead-level profile with a history of...SeniorRemote jobFull timeContract workFor contractors$204k - $259k
...other Waymo teams that consume map data. You will: Evaluate and launch machine learning models that automatically generate... ...work on various parts of the system ~ A passion for good software engineering We prefer: M.S. or Ph.D in Computer Science or a similar...SeniorFull timeRemote work- ...Role Overview Build and validate verifiable software engineering tasks and evaluation datasets derived from public repository histories, to measure and... ...the project. Qualifications Proven experience at a senior or tech-lead level working with high-quality public...SeniorFor contractorsLocal areaRemote work
- ...Role Overview Design and evaluate high-quality datasets and evaluations that advance large language models for code. You will work... ...detect errors and measure model capabilities across the software engineering lifecycle. Key Responsibilities Curate and author code...SeniorFor contractorsRemote work10 hours per weekFlexible hours
- ...experiences. About the Organization: The Evaluation team builds and evolves the evaluation... ...provide clear, explainable insights to engineering and leadership, including trend... ...design, code reviews, and by following software-engineering best practices. Interface with...SeniorLocal areaWork from home
- ...Role Overview Drive development of high-quality datasets and evaluation pipelines that improve and benchmark large language models for code generation and software engineering tasks. You will curate and author reference code, evaluate and refine AI-generated solutions...SeniorFull timeFor contractorsRemote work10 hours per weekFlexible hours
$40 per hour
...leading tech companies, focused on testing, evaluating, and improving AI systems.... ...NOT: Not data labeling. Not prompt engineering. Not writing code from scratch - the... ...evaluate. Qualifications ~5+ years in software development. ~Core stack: Python (FastAPI...SeniorPart time$180k - $240k
Role Description Deepgram is looking for a Senior Software Engineer - Model Evaluation & AI Systems to join the team responsible for validating the quality of our speech, audio, and multilingual models before they reach customers. This team owns the evaluation and quality...SeniorFull time$125k - $175k
...NinjaTrader equips traders with award-winning software and brokerage services to navigate... ...'ll do:We are seeking a Sr. Software Engineer to join our Evaluation Services team. Our Evaluation... ...same.In this role you will:Serve as the senior technical point of contact for...SeniorWork at officeRemote workWorldwideMonday to FridayFlexible hoursShift work$127k - $223k
...positive way. To learn more visit: The Evaluation Algorithms team is responsible for... ...highly realistic closed-loop simulation engine built with the latest in generative AI technologies... ...in Python programming and strong software engineering fundamentals with real-world...SeniorFull timeWork at officeWork from homeFlexible hours$40 per hour
...Role Overview Lead hands-on engineering work to create and validate verifiable software engineering tasks drawn from public repository histories, and evaluate how large language models perform on... ...software engineering experience at a senior or tech lead level, with...SeniorContract workFor contractorsLocal areaRemote work$50 per hour
...Role Overview Help build and evaluate the datasets and tooling that advance large language models for real-world software engineering. You will collaborate with researchers to curate code examples, author precise solutions, and correct AI-generated code across multiple...SeniorFull timeFor contractorsRemote work10 hours per weekFlexible hours$160k - $180k
...Greenfield Engineering, a Saalex Company, is seeking a Senior Software Engineer in Patuxent River, MD . This role involves applying advanced engineering principles to design, develop, and evaluate mission-critical military and aviation systems, ensuring compliance...SeniorFull time$85k - $100k
...Full-time Description Job Description: Software Engineer Position Overview The Software Engineer will provide critical engineering support in the design, development, testing, evaluation, and sustainment of software systems for Air Force and joint service...SeniorFull time$75 per hour
Role Description We are hiring senior full-stack engineers to help build and refine next-generation coding environments that evaluate AI agents. This work directly supports the development... ...~Senior-level experience in full-stack software engineering ~Deep professional...SeniorHourly payContract workRemote work- ...over sponsorship of an employment visa at this time. Senior Software Engineer In this role, you will: Lead moderately complex initiatives... ...complex technical challenges that require an in-depth evaluation of technologies and procedures Resolve moderately...SeniorWork experience placementRemote workWork visa
- ...billion dollar HR tech industry. Job Description The Senior Software Engineer works as a leader/key player on our development team to... ...Definition of requirements and design specifications # Evaluation of system integration requirements with other providers of...SeniorFull time
$140k - $175k
...development of the Gentuity imaging application software and internal software tools ·... ...features, GUI ideas, and for hardware evaluation. · Participate in evaluation and selection... ...of 5 years of experience in Software Engineering · Modern C++, including Objected...SeniorFull time$161.89k - $193.7k
...procedures for new assignments and select approaches through the evaluation and adaptation of complex techniques. Contribute to the... ...Implement REST services, XML, and XSD using MVC architecture. Software development and implementation using JavaScript, C#, jQuery...SeniorFull time- ...LiDAR, Radar) Build and maintain data collection, labeling and evaluation (metrics) pipeline Maintain proprietary multi-modal dataset... ...a deterministic secondary perception system Mentor junior engineers about best practices You have ~ Bachelor’s or Master’s...SeniorRemote jobFull time
- ...Overview \ \ We are seeking an experienced Software Engineer to develop, maintain, and enhance complex software systems supporting mission... ..., including hardware/software trade -off analysis. \\ Evaluate opportunities for software reuse and integration of Commercial...SeniorFull timeLocal area
- ...perception Build and maintain radar data collection, labeling, and evaluation pipelines Architect a deterministic secondary perception system with radar as a primary modality Mentor junior engineers on radar perception best practices You have ~ Bachelor's or...SeniorFull time
- ...well-structured design Partner closely with leadership and engineering to translate product goals into pragmatic solutions Help... ...infrastructure, developer tooling, agent frameworks, or replayable evaluation environments (e.g. Strandloop) ~ Strong communication and...SeniorRemote work
- ...smarter and more capable. The Role We’re looking for senior software engineers to help us realize TruckSmarter’s vision of eliminating... ...Stay ahead of emerging AI frameworks and technologies, evaluating and introducing them when they create real product advantage...SeniorFull timeWork at officeFlexible hours
$95k - $150k
...only the best from us. RSC2 is currently looking for a Senior Software Engineer to join our growing team in Aberdeen to support our SAI... ...Salary-Exempt We are an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion...SeniorFull timeContract workPart time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to (Senior) Software Engineer - Evaluation. Be the first to apply!
- agile software developer Remote
- software developer internship no experience Remote
- intermediate software engineer Remote
- software engineer staff Remote
- experienced software developer Remote
- work from home software developer Remote
- software developer no experience Remote
- software developer fintech Remote
- software data engineer Remote
- financial software developer Remote








