(Senior) Software Engineer - Evaluation
$120k - $250kPony Ai
Founded in 2016 in Silicon Valley, Pony.ai has quickly become a global leader in autonomous mobility and is a pioneer in extending autonomous mobility technologies and services at a rapidly expanding footprint of sites around the world. Operating Robotaxi, Robotruck and Personally Owned Vehicles (POV) business units, Pony.ai is an industry leader in the commercialization of autonomous driving and is committed to developing the safest autonomous driving capabilities on a global scale. Pony.ai’s leading position has been recognized, with CNBC ranking Pony.ai #10 on its CNBC Disruptor list of the 50 most innovative and disruptive tech companies of 2022. In June 2023, Pony.ai was recognized on the XPRIZE and Bessemer Venture Partners inaugural “XB100” 2023 list of the world’s top 100 private deep tech companies, ranking #12 globally. As of August 2023, Pony.ai has accumulated nearly 21 million miles of autonomous driving globally. Pony.ai went public at NASDAQ in November 2024.
Responsibility
- Design and build end-to-end data processing pipelines and toolchains to support large-scale data collection, batch processing, curation, and multi-dimensional evaluation.
- Design and implement high-performance backend services and distributed architectures , capable of handling large-scale real-time event processing.
- Collaborate across complex business components to lead the design of targeted system-level evaluation metrics, testing frameworks, and advanced simulation environments.
- Leverage and improve industry-leading, open-source distributed technologies to continuously advance infrastructure capabilities and processing efficiency.
- Drive critical architectural decisions and technology selection, ensuring rigorous standards for security, stability, scalability, and high performance.
- Establish and maintain comprehensive system monitoring, fine-grained observability metrics, and automated alerting frameworks to ensure low latency and reliable production operations.
- Build and optimize downstream engineering workflows for Large Language Models (LLMs), programmatically steering model reasoning, establishing guardrails, and mitigating hallucinations to ensure reliable outputs.
- Strong programming skills in C/C++, Python, and software design
- Strong foundation in mathematics and theoretical computer science, such as linear algebra, calculus, graph theory, computational geometry, combinatorial optimization algorithms, stochastic processes, and complexity analysis.
- Possessing solid engineering discipline, or demonstrating strong interest and potential in building large-scale systems, with the ability to maximize performance while keeping complexity and cost to a minimum.
- Pursueing a BS/MS or Ph.D in Computer Science or a related field
- Experience in large data set processing and familiarity with real time systems
- Solid experience in a fast-paced and structured engineering environment
- Full stack experience including both front end and back end is preferred
- Statistics analysis experience is preferred
Compensation and Benefits
Base Salary Range: $120,000 - $250,000 Annually
Compensation may vary outside of this range depending on many factors, including the candidate’s qualifications, skills, competencies, experience, and location. Base pay is one part of the Total Compensation and this role may be eligible for bonuses/incentives and restricted stock units.
Also, we provide the following benefits to eligible employees:
- Health Care Plan (Medical, Dental & Vision)
- Retirement Plan (Traditional and Roth 401k)
- Life Insurance (Basic, Voluntary & AD&D)
- Paid Time Off (Vacation & Public Holidays)
- Family Leave (Maternity, Paternity)
- Short Term & Long Term Disability
- Free Food & Snacks
$204k - $259k
...create a training ground for the Waymo Driver. The Simulator Evaluation team faces the ultimate data challenge: How do you... ...prove that a virtual world is "real"? We are looking for a Senior Software Engineer to build the metrics and systems that grade this hybrid environment...SeniorFull timeRemote work$224k - $356.5k
...the future of autonomous driving, and evaluation is how we know the drive is getting better... ...AI drivers!We are looking for a senior engineer to own the engine that makes it all trustworthy... ...experience).12+ years building software, with significant time in autonomous vehicles...SeniorFull timeRemote work- ...experiences. About the Organization The Evaluation team builds and evolves the evaluation... ...-level results into clear feedback for engineering and leadership, and help accelerate validated... ...to introspect autonomous driving software performance at subsystem interfaces across...SeniorFull timeLocal areaWork from home
$238k - $302k
...that are core to our autonomous driving software. We help our partners by offering the best... ...lifecycle, including training and evaluation. They are geared towards both scaling models... ...autonomous driving. We are looking for engineers with ML system expertise to help us...SeniorFull timeRemote work$204k - $259k
...Waymo Planner Evaluation Team Role Waymo is an autonomous driving technology company with... ...measuring and improving the quality of the software that drives the car. We are looking for experienced data-minded software engineers and data scientists to help us improve...SeniorFull timeRemote work$204k - $259k
...billions in simulation across 15+ U.S. states. Waymo's Release Evaluation org ensures that each version of the Waymo Driver is safe... ...objectives under resource constraints. Collaborate with other engineers, data scientists, statisticians and the leadership team to...SeniorFull timeRemote work$80 - $100 per hour
...Design and build the coding benchmarks and evaluation pipelines used to test frontier AI models on real software engineering work: Design coding benchmarks that evaluate... ...their country of residence. Nice to have Senior or Lead-level profile with a history of...SeniorRemote jobFull timeContract workFor contractors- ...and scale infrastructure for inference, evaluation, and continuous model improvement... ...with a strong blend of infrastructure engineering, production ML systems, LLM/LVM inference... ...background, especially in Python, with solid software engineering fundamentals. ~...SeniorFull timeWork at officeLocal areaFlexible hours3 days per week
- ...Role Overview Build and validate verifiable software engineering tasks and evaluation datasets derived from public repository histories, to measure and... ...the project. Qualifications Proven experience at a senior or tech-lead level working with high-quality public...SeniorFor contractorsLocal areaRemote work
- ...Role Overview Design and evaluate high-quality datasets and evaluations that advance large language models for code. You will work... ...detect errors and measure model capabilities across the software engineering lifecycle. Key Responsibilities Curate and author code...SeniorFor contractorsRemote work10 hours per weekFlexible hours
- ...Role Overview Drive development of high-quality datasets and evaluation pipelines that improve and benchmark large language models for code generation and software engineering tasks. You will curate and author reference code, evaluate and refine AI-generated solutions...SeniorFull timeFor contractorsRemote work10 hours per weekFlexible hours
- Role Description We're building a dataset to evaluate AI coding agents - how well a model handles real-world developer tasks. Essentially... ...scope and deeper involvement. Qualifications ~Strong software engineering experience: 8+ years in any modern programming language,...SeniorTemporary workFreelanceRemote workFlexible hours
- ...The Opportunity Deepgram is looking for a Senior Software Engineer - Model Evaluation & AI Systems to join the team responsible for validating the quality of our speech, audio, and multilingual models before they reach customers. This team owns the evaluation and...SeniorFull time
$125k - $175k
...NinjaTrader equips traders with award-winning software and brokerage services to navigate... ...'ll do:We are seeking a Sr. Software Engineer to join our Evaluation Services team. Our Evaluation... ...same.In this role you will:Serve as the senior technical point of contact for...SeniorWork at officeRemote workWorldwideMonday to FridayFlexible hoursShift work$125k - $175k
...starting out, NinjaTrader equips traders with award-winning software and brokerage services to navigate the world's leading... ...world. What you'll do: We are seeking a Sr. Software Engineer to join our Evaluation Services team. Our Evaluation Services business powers funded...SeniorFull timeWork at officeRemote workWorldwideMonday to FridayFlexible hoursShift work$40 per hour
...Role Overview Lead hands-on engineering work to create and validate verifiable software engineering tasks drawn from public repository histories, and evaluate how large language models perform on... ...software engineering experience at a senior or tech lead level, with...SeniorContract workFor contractorsLocal areaRemote work$50 per hour
...Role Overview Help build and evaluate the datasets and tooling that advance large language models for real-world software engineering. You will collaborate with researchers to curate code examples, author precise solutions, and correct AI-generated code across multiple...SeniorFull timeFor contractorsRemote work10 hours per weekFlexible hours$85k - $100k
...Full-time Description Job Description: Software Engineer Position Overview The Software Engineer will provide critical engineering support in the design, development, testing, evaluation, and sustainment of software systems for Air Force and joint service...SeniorFull time- ...billion dollar HR tech industry. Job Description The Senior Software Engineer works as a leader/key player on our development team to... ...Definition of requirements and design specifications # Evaluation of system integration requirements with other providers of...SeniorFull time
$140k - $175k
...development of the Gentuity imaging application software and internal software tools ·... ...features, GUI ideas, and for hardware evaluation. · Participate in evaluation and selection... ...of 5 years of experience in Software Engineering · Modern C++, including Objected...SeniorFull time- ...LiDAR, Radar) Build and maintain data collection, labeling and evaluation (metrics) pipeline Maintain proprietary multi-modal dataset... ...a deterministic secondary perception system Mentor junior engineers about best practices You have ~ Bachelor’s or Master’s...SeniorRemote jobFull time
- ...Overview \ \ We are seeking an experienced Software Engineer to develop, maintain, and enhance complex software systems supporting mission... ..., including hardware/software trade -off analysis. \\ Evaluate opportunities for software reuse and integration of Commercial...SeniorFull timeLocal area
- ...perception Build and maintain radar data collection, labeling, and evaluation pipelines Architect a deterministic secondary perception system with radar as a primary modality Mentor junior engineers on radar perception best practices You have ~ Bachelor's or...SeniorFull time
- ...Voltai is developing models and agents to evaluate, design, and interact with hardware and... ...keyword-match ~5+ years in backend engineering, full-stack development, or infrastructure... ...EDA, semiconductor, or hardware-adjacent software environments Our AI Bullshit Meter...SeniorFull timeInternshipWork at office
$95k - $150k
...only the best from us. RSC2 is currently looking for a Senior Software Engineer to join our growing team in Aberdeen to support our SAI... ...Salary-Exempt We are an equal opportunity employer. We evaluate qualified applicants without regard to race, color, religion...SeniorFull timeContract workPart time$160k - $210k
...businesses. We're seeking an experienced Engineer to develop and build innovative data... ...interactions with our data. Engage with software engineers, data scientists, and product... ...that move a customer from discovery to evaluation to purchase. Analyze and extract value...SeniorFull timeTemporary work$213k - $270k
...identify, and guide fixes of reliability and software integrity issues. We focus the... ...role you will report to a Staff Software Engineer / Tech Lead Manager. You will:... ...-scale cloud issues and their impact on evaluation vs. onroad behavior Automate measurement...SeniorFull timeRemote work- ...observability platform built on the Snowflake AI Data Cloud and engineered for scale. We ingest and store logs, metrics, traces, and... ...scale. Experience with query DSLs, CEL, or similar expression evaluation frameworks. Prior experience in a tech lead or staff engineer...SeniorFull time
$50 - $70 per hour
...Location: Remote Role Responsibilities Develop and complete coding tasks in various enterprise languages . Evaluate and improve systems and build toolchains using C and Rust . Enhance enterprise JVM and build ecosystems with C#...SeniorFull timeContract workSummer workRemote work- ...Department: IT Development Reports to: Software Engineering Manager Location: Hybrid - Dublin, Ohio (3 days) Remote (2 days) Full-... ...current with emerging technologies and industry trends. Evaluate frameworks, technology, platforms, and tooling solutions....SeniorFull timeRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to (Senior) Software Engineer - Evaluation. Be the first to apply!
- software engineer internship Remote
- software development engineer aws Remote
- software developer internship no experience Remote
- real time software engineer Remote
- financial software developer Remote
- oracle software engineer Remote
- part time software developer Remote
- graduate software developer Remote
- software engineer travel Remote
- experienced software developer Remote




