Staff Engineer, AI Benchmarking & Evaluation
Handshake
Handshake is hiring a Member of Technical Staff to define how frontier coding agents are benchmarked and improved. You will collaborate with top AI researchers and engineers to develop benchmarks, verifier systems, and data-quality techniques for coding applications. Early team members influence technical direction, culture, and open-source software. Location: San Francisco preferred; exceptional candidates in other locations are encouraged to apply.
#J-18808-LjbffrVacancy posted 19 hours ago
Similar jobs that could be interesting for youBased on the Staff Engineer, AI Benchmarking & Evaluation in New York, NY vacancy
- ...limitations, and training methods of frontier AI systems. Formulate research questions... ...post-training, model understanding, benchmarks, and AI evaluation. Develop research-grade datasets,... .... Collaborate with Research, Engineering, Product, and Operations teams to translate...Suggested
- ...SemiAnalysis LLC is hiring a Member of Technical Staff to develop training and inference benchmarks and system modelling for AI hardware. You will contribute to writing our newsletter articles with potential authorship recognition and collaborate with a global team of...Suggested
S27a
New York, NY18 hours ago - ...Waymo LLC seeks an experienced ML/AI specialist to lead cutting-edge deep learning and ML model development for evaluation systems powering autonomous driving. You will drive end-to-end ML initiatives, build Gen AI LLM/VLM solutions, and collaborate with cross-functional...Suggested
Waymo
New York, NY2 days ago $115k - $260k
...Culture, Great Rewards, and Great Careers. Sr. Staff AI Engineer, AI Agent Platform (Agent & Evaluation Harness) Why Join GEICO? GEICO is transforming... ...compare evaluations for their agents, spanning offline benchmarks, simulated users and environments, and online...SuggestedHourly payWork experience placementLocal areaGovernment Employees Insurance Company
New York, NY2 days ago$238k - $302k
....S. states. The Driver Understanding and Evaluation (DUE) team at Waymo is developing rich metrics... ...are looking for researchers and software engineers who are passionate about developing... ...high-volume workflows Design and build Gen AI LLM/VLM solutions for self driving car...SuggestedFull timeWaymo
New York, NY2 days ago$265k - $331k
About Scale AIScale AI is the data foundation for AI, helping... ...one of the hardest engineering challenges.As a Staff Frontier Agent Engineer (Applied... ...customers to design, evaluate, and deploy intelligent systems... ...frameworks using offline benchmarks, online A/B experiments,...Full timeScale AI
New York, NY7 days ago$234k
We’re looking for a Staff Software Engineer with deep experience in GenAI/ML to... ...lead efforts to design, train, evaluate, and deploy GenAI/ML models... ...to production.Build and benchmark GenAI/ML models using state-... ...and security platform for the AI era, providing businesses with...Work at officeDatadog
New York, NY6 days ago- ...accelerate how the world engineers new hardware. Nominal'... ...Anduril, Shield AI, Hermeus, Albedo, Shinkei... ...The Role: As a Staff Research Engineer supporting... ...tasks. Invent new benchmarks for agents working... .... ~ Deep experience evaluating LLM or agent systems,...Permanent employment
Nominal
New York, NY16 hours ago - ...frameworks, job queues, databases, AI agent tools, or a platform they... .... We are looking for a Senior Staff Product Marketing Engineer to focus on competitive benchmarking and positioning and message.... ...research, hands‑on product evaluation, and reproducible benchmarks. You...Shift work
$400k
...Remote | Member of Technical Staff, Research Engineering — $400,000–$800,000/year... ...ML-oriented data systems, evaluation infrastructure, and... ...to contribute to advanced AI research and development.... ...model performance Develop benchmarking frameworks measuring capability...Full timeRemote work24-Mag Llc
New York, NY3 days ago- ...developers nationwide. We combine AI-powered technology with an in... ...of registered architects, engineers, and city planners to deliver... ...where appropriate. Evaluation & safety harness: Stand up task... ...; familiarity with agent benchmarks (and their pitfalls) and how...Full timeSummer workWork at officeFlexible hours
GreenLite
New York, NY1 day ago - ...helping companies discover, evaluate, and optimize their event strategy... ...by delivering proprietary, AI-driven insights across... ...for customers while receiving benchmarking and audience intelligence in... ...predictable and scalable growth engine. In addition, Vendelux Meetings...
Vendelux
New York, NY4 days ago - ...one with the other. Crosby is the first AI-native law firm, helping ambitious... ...startup GC) and John (Penn, first 15 engineers at Ramp). Help us transform one of society... .... You'll turn validated methods, benchmarks, and evaluation infrastructure from our research team...Contract workWork at officeImmediate start
C R O S B Y
New York, NY4 days ago - ...intersection of research and deployment. You will own end-to-end evaluation workflows for open models, turning signals into measurable... ...collection, and live deployments. Work closely with researchers, engineers, and partners to scope data, build pipelines, and drive...
Reflection
New York, NY2 days ago - ...building and running the world’s best data and AI infrastructure platform so our customers... ...to improve their business. Founded by engineers — and customer obsessed — we leap at... ...production at enterprise scale, including evaluation, cost and performance tuning, and the security...Worldwide
DataBricks
New York, NY6 days ago - ...clinical compiler, already live in Canadian public health care today. The role involves building deterministic validators, co-designing benchmarks, and working directly with the founders and clinicians to translate model failures into improved system #J-18808-Ljbffr...Live in
OpenKrill
New York, NY3 days ago $160k - $300k
...lead to own quality for the AI that powers Solstice. Our platform... ...This is a senior, hands‑on engineering job. Most of your time goes... ...ll be working on Build our evaluation systems. Because we can't... ...as part of the early team. We benchmark against top‑tier startups and...Permanent employmentH1bVisa sponsorshipRelocation packageNight shiftSolstice
New York, NY4 days ago$180k - $350k
...Hume AI is seeking talented researchers and engineers interested in working with our AI research team to build state... ...audio understanding models and evaluation platforms for enterprises.... ...model training, inference, and benchmarking as well as massive‑scale data collection...Hume
New York, NY16 hours ago$220k - $330k
...and applications that power today's AI stack using sustainable technology... ...the Role Nscale is looking for a Staff AI Engineer (Specialised) to set technical... ...NVLink-scale systems Evals and benchmarking: designing evaluation frameworks, turning customer requirements...Flexible hoursNscale Limited
New York, NY4 days ago$217k - $303.9k
...infrastructure, product engineering, and user experience -... ...We are looking for a Staff Site Reliability... ...and country location, benchmarked against similar stage... ...artificial intelligence (AI). You will have the opportunity... ...this information to evaluate your application for...For contractorsWork experience placementFlexible hoursReddit, Inc.
New York, NY4 days ago- ...we are not just building AI features, we are building... ...surface at Rippling.As a Staff Machine Learning Engineer, you will own the end-to-end... ...strategy, model development, evaluation, and production deployment... ...infrastructure: offline benchmarks, A/B testing, and continuous...Work at office3 days per week
Rippling
New York, NY6 days ago $269.1k - $307.2k
## Staff Engineer -Developer Insights & AI TelemetryApply: New York, NY: McLean, VA: Richmond, VA: Plano, TX: Wilmington, DE: Full time: Posted Today... ...direct technical contribution—prototyping systems, evaluating build vs. buy paths, authoring code patterns, and deeply...Full timePart timeLocal areaShift workCapital One Group
New York, NY4 days ago$269.1k - $307.2k
...As a Staff Data Engineer at Capital One, you will be part of a community of technical experts working... ...enterprise roadmap for distributed AI telemetry, streaming data engineering,... ...technical contribution—prototyping systems, evaluating build vs. buy paths, authoring code...Full timePart timeLocal areaShift workCapital One
New York, NY3 days ago$210k - $234k
...service to seller and buyer clients. Engineering @ Compass Compass has built the first... ...Artificial Intelligence" that cloud, mobile and AI technologies enable. We are building AI... ...reusable libraries, CI/CD checks, and evaluation harnesses so secure AI patterns are the...Minimum wageFlexible hoursCompass, Inc.
New York, NY2 days ago$217k - $288k
...the secure path - the easy path for our engineering org of roughly 145 engineers, baking secure... ...it, including for our fast growing AI and agent features. What You’ll Be Doing... ..., explain how it works and what it evaluates, and obtain consent beforehand. New York...Full timeRemote workHome officeFlexible hoursDay shift3 days per weekGrow Therapy
New York, NY4 days ago- ...are building Wall Street's first true AI analyst. Our mission is to empower finance... ...you to join us. The Role As a Staff Security Engineer at Rogo, you'll play a key role in... ...security frameworks such as SOC 2, CIS Benchmarks, ISO 27001/42001, or NIST CSF. Bonus...
ScOp Venture Capital LLC.
New York, NY4 days ago $192k - $260k
...insights to improve their business. Founded by engineers — and customer obsessed — we leap at... ..., to bring enterprise-grade ML and AI personalization to every company whose data... ...leader like Databricks.The impact you'll have:Evaluate ML and LLM approaches for CustomerLake's...Work at officeLocal areaWorldwideDataBricks
New York, NY6 days ago$201.3k - $352.3k
...DescriptionIt all started when engineer Fred Luddy wrote code that... ...work. Today, ServiceNow is the AI control tower for business reinvention... ...like. The role As a Senior Staff Engineer, you own the... ...is a plus. Experience with AI evaluation, safety, governance, or policy...Work at officeImmediate startRemote workFlexible hoursServiceNow
New York, NY4 days ago$180k - $250k
CoreWeave is The Essential Cloud for AI. Built for pioneers by... ..., and production software engineering to help engineers and customers... ...underlying ML systems, services, evaluation frameworks, and product... ...environments.About the roleAs a Staff Machine Learning Engineer,...Permanent employmentFull timeTemporary workCasual workWork at officeFlexible hoursCoreWeave
New York, NY5 days ago$170k - $225k
...Taskrabbit, and we're looking for a Staff Machine Learning Engineer to join our team and lead the next phase... ...engineering and training through evaluation, deployment, monitoring, and optimization... ...consistent with applicable law. AI-Assisted Prescreening Notice [US Based...Work at officeImmediate startFlexible hoursTaskrabbit
New York, NY5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Engineer, AI Benchmarking & Evaluation. Be the first to apply!
Related searches
- project engineer assistant project manager New York, NY
- staff security engineer New York, NY
- staff engineer New York, NY
- research assistant engineering New York, NY
- senior staff systems engineer New York, NY
- engineering aide New York, NY
- senior staff engineer New York, NY
- assistant engineering manager New York, NY
- assistant chief engineer New York, NY
- staff data engineer New York, NY