Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Frontier Benchmarking Research Engineer

Refresh

Refresh is seeking a Research Engineer focused on benchmarking, to build rigorous evaluations of real-world coding and computer-use capabilities. You will translate expert workflows into verifiable tests and run benchmarks against frontier models. In this role you’ll work across our core stack (Vercel, Supabase, Render) and engage with reinforcement learning and supervised fine-tuning at a high level to publish robust numbers. #J-18808-Ljbffr Refresh

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Frontier Benchmarking Research Engineer in San Francisco, CA vacancy
  • # Research Engineer, BenchmarkingEngineeringSan FranciscoFull-timeBuild the benchmarks frontier labs use to measure real-world coding and computer-use capability. Translate expert workflows into rigorous, verifiable evaluations, run them against frontier models, and publish... 
    Suggested

    Refresh

    San Francisco, CA
    3 days ago
  • $250k - $400k

     ...Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality datasets,...  ...reinforcement learning environments, and frontier research benchmarks that improve model capabilities in software engineering, enterprise knowledge work, and advanced STEM... 
    Suggested

    Turing

    San Francisco, CA
    25 days ago
  •  ...build the infrastructure that enables frontier AI labs and enterprises to train, evaluate...  ...own expertise and standards. Our APEX benchmarks measure how AI performs on...  ...and London. About the Role As a Research Engineer at Mercor, you’ll work at the intersection... 
    Suggested
    Internship
    Local area
    Relocation package

    mercor

    San Francisco, CA
    5 days ago
  • SuperAnnotate AI, Inc. seeks a Research Engineer to drive frontier AI research from direction to MVPs, including human annotation iterations and dataset creation. You will collaborate with strategic leads to scope work, implement methods, and produce tangible outputs such... 
    Suggested

    SuperAnnotate AI, Inc.

    San Francisco, CA
    1 day ago
  •  ...Research Engineer On Physical AI TeamHedra is a pioneering generative modeling company — first...  ...be direct. If you want to work at the frontier of generative modeling and physical AI...  ...sequencesEvaluate model performance using both benchmark datasets and real-world deployment... 
    Suggested

    HEDRA INC

    San Francisco, CA
    2 days ago
  •  ...by Amplify and CRV, and most frontier labs are already customers....  ...’ll develop Taste’s internal research, evaluation design, judges and...  ...Develop frontier evaluations and benchmarks for subjective domains....  ...a researcher, moves like an engineer. Creative, scrappy, comfortable... 

    Taste Labs

    San Francisco, CA
    4 days ago
  •  ...will: Design multimodal datasets that directly shape what frontier world models learn Run controlled training experiments to understand...  ..., filtering, and quality control Develop evaluations and benchmarks that measure meaningful model improvements Why join:... 

    Goliath Partners LP

    San Francisco, CA
    5 days ago
  •  ...Ando Research Team MemberAndo is a messaging platform where AI agents take on work alongside...  ...Evaluating that judgment means building benchmarks where the ground truth includes silence....  ...in-house. Part of the job is evaluating frontier vendors and research teams (eval... 
    Work from home

    ASARI S.A de CV

    San Francisco, CA
    3 days ago
  • $180k - $250k

     ...Research EngineerSuperAnnotate helps the world's leading AI teams...  ...to keep pace with a wave of frontier-facing work: internal research...  ...of the field. As a Research Engineer, you'll take a research direction...  ...– finding the right papers, benchmarks, and prior work, reimplementing... 
    Full time

    SuperAnnotate AI

    San Francisco, CA
    2 days ago
  •  ...Research EngineerNew York - Hybrid; San Francisco Bay Area - HybridAbout InvisibleInvisible...  ...organization, focused on evaluation methodology, benchmarks, and RL environments for frontier labs and enterprise clients. As a Research Engineer, you'll work on how frontier models are... 
    Full time
    Work at office
    Local area
    Remote work

    Invisible Technologies Inc. Defunct

    San Francisco, CA
    5 days ago
  •  ...Research EngineerDatacurve provides the frontier coding data that powers the world's most advanced models. We absorb...  ...world's first autonomous data engine, allowing us to teach the next generation...  ...domain. You will produce the benchmarks, artifacts, and technical... 
    Shift work

    Datacurve

    San Francisco, CA
    5 days ago
  •  ...ElevenlabsElevenLabs is an AI research and product company...  ...lives. We are researchers, engineers, and operators. IOI medalists...  ...ElevenLabs. You'll work at the frontier of audio AI, shaping the models...  ...paradigms.Designing rigorous benchmarks and evaluations that tell us... 
    Immediate start
    Remote work

    Eleven Labs

    San Francisco, CA
    1 day ago
  •  ...have become a massive tax of engineering velocity. Resolve AI is...  ...workflows end-to-end, balancing research and engineering to create production...  ...and execute benchmarks to evaluate AI models, improve...  ...Future: Help build the next frontier in enterprise software and define... 
    Work at office

    Resolve AI

    San Francisco, CA
    2 days ago
  •  ...Role Description This is a Research Engineer role focused on post‑training and reasoning. Full...  ...contributions, or production ML systems at a lab/frontier company Comfortable working with a...  ...from our expert network Building benchmarks and evals that meaningfully measure... 
    Full time
    Part time
    Remote work
    Flexible hours

    Cobalt.io

    San Francisco, CA
    1 day ago
  • $200k - $225k

     ...AlembicAlembic is where top engineers are solving marketing's hardest...  .... If you're looking for frontier technical challenges at an applied...  ...advanced analyticsDocument research and implementation decisions...  ...earn at or above real-time benchmarks. Ownership: Our generous... 

    Alembic

    San Francisco, CA
    1 day ago
  • $150k - $250k

     ...critical operations for the frontier of AI. Our customers include...  ...global social organizations.We research and deploy technologies that...  ...Looking ForAt Distyl, Research Engineers build the bridge between...  ...changes, not just whether a benchmark improvesAI Systems Mindset: You... 
    Work at office
    3 days per week

    Distyl AI

    San Francisco, CA
    1 day ago
  •  ...Google Labs.Sierra’s Applied Research team advances the quality,...  ...to use externally provided frontier intelligence, balancing quality...  ...to applied ML research or benchmarks.Our valuesTrust: We build trust...  ...timeLocation TypeOn-siteDepartmentEngineeringPlatform Engineering
    Full time
    Local area
    Flexible hours

    Sierra

    San Francisco, CA
    4 days ago
  • $207k - $300k

     ...and introspection probes.Collaborate with safety engineering to develop new capabilities based on adversarial research.Contribute to increasing Gemini’s capabilities...  ...world from misuse of the next generation of frontier models and help defend against emerging cyber security... 

    Google

    San Francisco, CA
    1 day ago
  •  ...build the infrastructure that enables frontier AI labs and enterprises to train, evaluate...  ...own expertise and standards. Our APEX benchmarks measure how AI performs on economically...  ...frontier of world-building, verifier engineering, and more alongside our partners. Your... 
    Full time
    Local area
    Relocation package

    mercor

    San Francisco, CA
    5 days ago
  •  ...a quickly growing group of committed researchers, engineers, policy experts, and business leaders...  ...view of how RL training behaves at the frontier because the system you own sits...  ...through profiling, optimization, and benchmarking Make the system easier for researchers... 
    Work at office
    Local area
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    9 days ago
  • $260k - $350k

     ...Headquartered in San Francisco, Turing works with frontier AI labs to generate high-quality datasets,...  ...reinforcement learning environments, and frontier research benchmarks that improve model capabilities in software engineering, enterprise knowledge work, and advanced STEM... 

    Turing

    San Francisco, CA
    8 days ago
  • $150 per hour

     ...technical talent with leading AI research labs. Headquartered in San...  ..., our investors include Benchmark , General Catalyst , Peter...  ...machine learning engineering tasks for a long-horizon AI...  ...Design tasks that challenge frontier models, ensuring they are solved... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  •  ...build the infrastructure that enables frontier AI labs and enterprises to train, evaluate...  ...own expertise and standards. Our APEX benchmarks measure how AI performs on...  ...and London. About the Role As a Research Engineer at Mercor, you’ll work at the intersection... 
    Internship
    Remote work
    Relocation package

    mercor

    San Francisco, CA
    5 days ago
  • Phinity is an AI research lab focused on solving chip design. The bottleneck...  .... We are hiring a Research Engineer to develop data in order to...  ...tasks Profile and benchmark unreleased models Run and experiment...  ...Phinity Work directly with frontier AI labs to re-imagine ASIC... 
    Work at office

    Phinity Labs

    San Francisco, CA
    1 day ago
  •  ...the design suite for molecules. We train frontier models that learn the underlying...  ...sciences the same way it reinvented software engineering, and Chai is at the forefront of this shift...  ....About the RoleWe are seeking an AI Research Engineer to help design, train, evaluate... 
    Shift work

    Chai Discovery

    San Francisco, CA
    2 days ago
  •  ...Join AfterQueryAfterQuery is an applied research lab curating data solutions for foundation model development.We serve every frontier AI lab with the mission of delivering the...  ...you'll have the opportunity to shape the engineering organization and lead major technical initiatives... 
    Local area

    AfterQuery

    San Francisco, CA
    1 day ago
  • $180k - $290k

     ...Our FLUX models top public benchmarks and run in production in some...  ...We're a small, fast growing research lab competing with the largest...  ...into “research” or “engineering.” A promising architecture only...  ...or co-owned training for a frontier foundation model that shipped... 
    Full time
    Work at office
    2 days per week

    Black Forest Labs

    San Francisco, CA
    9 days ago
  • $250k - $400k

     ...Research Engineer – Post-Training, Agents, Evals Fundamental AI Research Institute San Francisco Bay Area, USA (hybrid) $250,000 –...  ...including developing agentic and evaluation harnesses Develop benchmarks and evaluations for reasoning and agentic capabilities... 
    Relocation package

    Storm3

    San Francisco, CA
    2 days ago
  •  ...David AIDavid AI is the first audio data research company. We bring an R&D approach to...  ...founded in 2024 by a team of former Scale AI engineers and operators. In less than a year, we'...  ...to join us on our mission to push the frontier of audio AI.About Our Research TeamOur Research... 
    Work at office

    David AI

    San Francisco, CA
    4 days ago
  • $275k

     ...promising path to safe AGI lies in automating research and code generation to improve models...  ...humans can alone. Our approach combines frontier-scale pretraining, domain-specific RL,...  ...this goal.About the RoleAs a Research Engineer, you'll work on training, evaluating, and... 
    Relocation

    CapitalG

    San Francisco, CA
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Frontier Benchmarking Research Engineer. Be the first to apply!