Frontier Benchmarking Research Engineer
Refresh
Refresh is seeking a Research Engineer focused on benchmarking, to build rigorous evaluations of real-world coding and computer-use capabilities. You will translate expert workflows into verifiable tests and run benchmarks against frontier models. In this role you’ll work across our core stack (Vercel, Supabase, Render) and engage with reinforcement learning and supervised fine-tuning at a high level to publish robust numbers. #J-18808-Ljbffr Refresh
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Frontier Benchmarking Research Engineer in San Francisco, CA vacancy
- # Research Engineer, BenchmarkingEngineeringSan FranciscoFull-timeBuild the benchmarks frontier labs use to measure real-world coding and computer-use capability. Translate expert workflows into rigorous, verifiable evaluations, run them against frontier models, and publish...Suggested
$250k - $400k
...Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality datasets,... ...reinforcement learning environments, and frontier research benchmarks that improve model capabilities in software engineering, enterprise knowledge work, and advanced STEM...Suggested- ...build the infrastructure that enables frontier AI labs and enterprises to train, evaluate... ...own expertise and standards. Our APEX benchmarks measure how AI performs on... ...and London. About the Role As a Research Engineer at Mercor, you’ll work at the intersection...SuggestedInternshipLocal areaRelocation package
- SuperAnnotate AI, Inc. seeks a Research Engineer to drive frontier AI research from direction to MVPs, including human annotation iterations and dataset creation. You will collaborate with strategic leads to scope work, implement methods, and produce tangible outputs such...Suggested
- ...Research Engineer On Physical AI TeamHedra is a pioneering generative modeling company — first... ...be direct. If you want to work at the frontier of generative modeling and physical AI... ...sequencesEvaluate model performance using both benchmark datasets and real-world deployment...Suggested
- ...by Amplify and CRV, and most frontier labs are already customers.... ...’ll develop Taste’s internal research, evaluation design, judges and... ...Develop frontier evaluations and benchmarks for subjective domains.... ...a researcher, moves like an engineer. Creative, scrappy, comfortable...
- ...will: Design multimodal datasets that directly shape what frontier world models learn Run controlled training experiments to understand... ..., filtering, and quality control Develop evaluations and benchmarks that measure meaningful model improvements Why join:...
- ...Ando Research Team MemberAndo is a messaging platform where AI agents take on work alongside... ...Evaluating that judgment means building benchmarks where the ground truth includes silence.... ...in-house. Part of the job is evaluating frontier vendors and research teams (eval...Work from home
$180k - $250k
...Research EngineerSuperAnnotate helps the world's leading AI teams... ...to keep pace with a wave of frontier-facing work: internal research... ...of the field. As a Research Engineer, you'll take a research direction... ...– finding the right papers, benchmarks, and prior work, reimplementing...Full time- ...Research EngineerNew York - Hybrid; San Francisco Bay Area - HybridAbout InvisibleInvisible... ...organization, focused on evaluation methodology, benchmarks, and RL environments for frontier labs and enterprise clients. As a Research Engineer, you'll work on how frontier models are...Full timeWork at officeLocal areaRemote work
- ...Research EngineerDatacurve provides the frontier coding data that powers the world's most advanced models. We absorb... ...world's first autonomous data engine, allowing us to teach the next generation... ...domain. You will produce the benchmarks, artifacts, and technical...Shift work
- ...ElevenlabsElevenLabs is an AI research and product company... ...lives. We are researchers, engineers, and operators. IOI medalists... ...ElevenLabs. You'll work at the frontier of audio AI, shaping the models... ...paradigms.Designing rigorous benchmarks and evaluations that tell us...Immediate startRemote work
- ...have become a massive tax of engineering velocity. Resolve AI is... ...workflows end-to-end, balancing research and engineering to create production... ...and execute benchmarks to evaluate AI models, improve... ...Future: Help build the next frontier in enterprise software and define...Work at office
- ...Role Description This is a Research Engineer role focused on post‑training and reasoning. Full... ...contributions, or production ML systems at a lab/frontier company Comfortable working with a... ...from our expert network Building benchmarks and evals that meaningfully measure...Full timePart timeRemote workFlexible hours
$200k - $225k
...AlembicAlembic is where top engineers are solving marketing's hardest... .... If you're looking for frontier technical challenges at an applied... ...advanced analyticsDocument research and implementation decisions... ...earn at or above real-time benchmarks. Ownership: Our generous...$150k - $250k
...critical operations for the frontier of AI. Our customers include... ...global social organizations.We research and deploy technologies that... ...Looking ForAt Distyl, Research Engineers build the bridge between... ...changes, not just whether a benchmark improvesAI Systems Mindset: You...Work at office3 days per week- ...Google Labs.Sierra’s Applied Research team advances the quality,... ...to use externally provided frontier intelligence, balancing quality... ...to applied ML research or benchmarks.Our valuesTrust: We build trust... ...timeLocation TypeOn-siteDepartmentEngineeringPlatform EngineeringFull timeLocal areaFlexible hours
$207k - $300k
...and introspection probes.Collaborate with safety engineering to develop new capabilities based on adversarial research.Contribute to increasing Gemini’s capabilities... ...world from misuse of the next generation of frontier models and help defend against emerging cyber security...- ...build the infrastructure that enables frontier AI labs and enterprises to train, evaluate... ...own expertise and standards. Our APEX benchmarks measure how AI performs on economically... ...frontier of world-building, verifier engineering, and more alongside our partners. Your...Full timeLocal areaRelocation package
- ...a quickly growing group of committed researchers, engineers, policy experts, and business leaders... ...view of how RL training behaves at the frontier because the system you own sits... ...through profiling, optimization, and benchmarking Make the system easier for researchers...Work at officeLocal areaVisa sponsorshipFlexible hours
$260k - $350k
...Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality datasets,... ...reinforcement learning environments, and frontier research benchmarks that improve model capabilities in software engineering, enterprise knowledge work, and advanced STEM...$150 per hour
...technical talent with leading AI research labs. Headquartered in San... ..., our investors include Benchmark , General Catalyst , Peter... ...machine learning engineering tasks for a long-horizon AI... ...Design tasks that challenge frontier models, ensuring they are solved...Contract workSummer workRemote work- ...build the infrastructure that enables frontier AI labs and enterprises to train, evaluate... ...own expertise and standards. Our APEX benchmarks measure how AI performs on... ...and London. About the Role As a Research Engineer at Mercor, you’ll work at the intersection...InternshipRemote workRelocation package
- Phinity is an AI research lab focused on solving chip design. The bottleneck... .... We are hiring a Research Engineer to develop data in order to... ...tasks Profile and benchmark unreleased models Run and experiment... ...Phinity Work directly with frontier AI labs to re-imagine ASIC...Work at office
- ...the design suite for molecules. We train frontier models that learn the underlying... ...sciences the same way it reinvented software engineering, and Chai is at the forefront of this shift... ....About the RoleWe are seeking an AI Research Engineer to help design, train, evaluate...Shift work
- ...Join AfterQueryAfterQuery is an applied research lab curating data solutions for foundation model development.We serve every frontier AI lab with the mission of delivering the... ...you'll have the opportunity to shape the engineering organization and lead major technical initiatives...Local area
$180k - $290k
...Our FLUX models top public benchmarks and run in production in some... ...We're a small, fast growing research lab competing with the largest... ...into “research” or “engineering.” A promising architecture only... ...or co-owned training for a frontier foundation model that shipped...Full timeWork at office2 days per week$250k - $400k
...Research Engineer – Post-Training, Agents, Evals Fundamental AI Research Institute San Francisco Bay Area, USA (hybrid) $250,000 –... ...including developing agentic and evaluation harnesses Develop benchmarks and evaluations for reasoning and agentic capabilities...Relocation package- ...David AIDavid AI is the first audio data research company. We bring an R&D approach to... ...founded in 2024 by a team of former Scale AI engineers and operators. In less than a year, we'... ...to join us on our mission to push the frontier of audio AI.About Our Research TeamOur Research...Work at office
$275k
...promising path to safe AGI lies in automating research and code generation to improve models... ...humans can alone. Our approach combines frontier-scale pretraining, domain-specific RL,... ...this goal.About the RoleAs a Research Engineer, you'll work on training, evaluating, and...Relocation
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Frontier Benchmarking Research Engineer. Be the first to apply!
Related searches
- junior machine learning research engineer San Francisco, CA
- senior research engineer San Francisco, CA
- research engineer San Francisco, CA
- deep learning research engineer San Francisco, CA
- research programmer San Francisco, CA
- research assistant engineering San Francisco, CA
- research software engineer San Francisco, CA
- ai research engineer San Francisco, CA
- research assistantship San Francisco, CA
- research writer San Francisco, CA




