Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Research Engineer, Model Evaluation and Improvement

$136.44k - $166.75k

Benchling

We are rebuilding biotech for the AI era.When a breakthrough is delayed, the world waits. Getting a molecule from discovery to patients, or a crop from lab to field, involves thousands of slow, manual, disconnected steps. AI has the potential to change this, compressing decades of R&D work into years. But that only happens when clean, structured scientific data and AI are built into how science gets done.Benchling is the AI platform for biotech R&D. Scientists use Benchling to design experiments, capture structured data, and run AI agents and models directly in their workflows. Over 200,000 scientists around the world trust Benchling to power their most important work, from academic labs to Sanofi, Moderna, and more than half of the world's top 50 biopharma.We’re building an AI scientist for our customers. We can’t do that if we haven’t built the muscle ourselves. AI fluency is the foundation we build on; it's core to how we work, and we're committed to helping every new hire integrate it into their day-to-day. As part of our interview process, you'll complete a brief AI-focused exercise or discussion so we can understand how you think about and use AI to drive impact in your role. Feel free to reference any tools, platforms, or workflows you use today.ROLE OVERVIEWWe’re a team focused on making frontier AI models better at science. LLMs know an extraordinary amount of biology, but there’s still a large gap in reasoning for the real-world problems scientists face every day. We recently published some of our workhere.You’ll build the datasets, evaluations, and systems that help close that gap. You’ll work with scientists to turn complex scientific work into rigorous tasks that models can learn from and be evaluated against. You’ll partner with leading AI labs to understand where models fail and how to improve them.This is an early and rapidly evolving area. You’ll work at the intersection of software engineering, biology, and frontier AI: finding tasks that are challenging for LLMs and valuable to scientists, designing evaluations that capture real scientific judgment, and building systems to create these tasks at scale.RESPONSIBILITIESBuild datasets for evaluating and improving frontier models, turning complex scientific data into high-quality tasks and environments for LLMs.Analyze model failure modes, running experiments across frontier models to understand where they struggle and identify opportunities for improvement.Build scalable data infrastructure, creating pipelines that curate, transform, and validate large volumes of scientific data into tasks for model evaluation and improvement.Collaborate with frontier AI labs, helping develop and evaluate new approaches for improving models on challenging scientific tasks.Work closely with scientists, translating expert judgment into problems and evaluation criteria that can reliably distinguish strong model behavior.QUALIFICATIONS2+ years at the intersection of biology and AI, with academic or industry experience evaluating and improving scientific models or LLMs for biological applications.Experience building with LLMs, with an intuition for where current models excel, where they struggle, and how to design systems around their capabilities.Curiosity and excitement about frontier AI, with a desire to understand and push the capabilities of rapidly improving models.Comfort working on ambiguous problems, where the playing field is rapidly shifting and the right technical approaches are still being discovered.Collaborative mindset, able to work closely with engineers, scientists, and external research partners.Desire to work in a fast-paced environment, where priorities can shift and rapid experimentation is encouraged.HOW WE WORKWe prioritize in-office collaboration. Employees are expected to be on-site 5 days per week (Monday - Friday).#LI-KWBenchling welcomes everyone.We believe diversity enriches our team so we hire people with a wide range of identities, backgrounds, and experiences. We are an equal opportunity employer. That means we don’t discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. We also consider for employment qualified applicants with arrest and conviction records, consistent with applicable federal, state and local law, including but not limited to the San Francisco Fair Chance Ordinance.Compensation Range: $136,435 - $166,754LocationSan Francisco, CAEmployment TypeFull timeLocation TypeOn-siteDepartmentEngineeringCompensation$136,435 – $166,754 • Offers EquityU.S. BenefitsFull-time U.S. employees enjoy a comprehensive benefits program including equity, health, dental, vision, 401(k)+ employer match, wellness, commuter, and more.Benchling takes a market-based approach to pay. The candidate's starting pay will be determined based on job-related skills, experience, qualifications, interview performance, and work location.

Vacancy posted 23 hours ago
Similar jobs that could be interesting for youBased on the Research Engineer, Model Evaluation and Improvement in San Francisco, CA vacancy
  • Reflection, a research lab, seeks to advance open and accessible intelligence. You will...  ...rigorous comparative analyses, build evaluation systems, and develop frameworks that...  ...applied teams to translate insights into model improvements. The role emphasizes high agency in a... 
    Suggested

    Reflection AI

    San Francisco, CA
    5 days ago
  •  ...generation multimodal AI models and a proprietary, high-...  ...-on support from AMD engineers the team is scaling rapidly...  ...About the role As a Research Engineer focused on Model Evaluation & MLOps, you will build...  ...performance benchmarking, and improve the MLOps workflows that... 
    Suggested
    Full time
    Flexible hours

    Sciforium

    San Francisco, CA
    9 days ago
  • $70 - $90 per hour

     ...Research Engineer Multimodal AI & Applied Machine Learning This fully remote Research Engineer role focuses on improving the data, evaluation systems, and production workflows behind applied AI models. You will work across research and engineering, improving dataset... 
    Suggested
    Contract work
    Temporary work
    Remote work
    Daly City, CA
    1 day ago
  •  ...technology that directly improves patient outcomes....  ...daily U.S. X-rays. Our research team brings experience...  ...of vision-language models for medical imaging....  ...team of researchers and engineers, with the freedom to pursue...  ...distributed training, evaluation, and handoff to post-... 
    Suggested
    Full time

    epsilon-health

    San Francisco, CA
    9 days ago
  •  ...unifies compute, environments, evaluations, secure sandboxes, high-...  ..., and continuously improving production models. We are building open frontier...  ...horizon tasks like autonomous research, and the full-stack...  ...Requirements Strong software engineering skills in Python and... 
    Suggested
    Full time
    Work at office
    Remote work
    Visa sponsorship
    Relocation package
    Flexible hours

    PrimeIntellect

    San Francisco, CA
    9 days ago
  • The Role As a Research Engineer - Model Architectures , you will be a core contributor to Zyphra’s AI Architecture Research Team. This will involve...  ...architectures and training methodologies, with a focus on improving core modeling capabilities (e.g., loss per flop or loss... 
    Work at office
    Relocation package

    Zyphra Technologies Inc.

    San Francisco, CA
    4 days ago
  •  ...advancing large language models (LLMs) and autonomous AI agents...  ...sophisticated datasets, evaluation methodologies, and data...  ...infrastructure that help leading AI research laboratories improve model reasoning, behavior,...  ...talented researchers, engineers, and commercial teams to... 
    Immediate start

    Aperio Talent Solutions

    San Francisco, CA
    8 days ago
  • $315k - $340k

     ...Research Engineer, Production Model Post Training San Francisco, CA | New York City, NY | Seattle, WA About...  ..., implementing, scaling, and improving post-training techniques like Constitutional...  ...for model fine-tuning and evaluation Develop tools to measure and improve... 
    Full time
    Contract work
    For contractors
    For subcontractor
    Work at office
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    3 days ago
  •  ...Research Engineer, Model Evaluations Remote-Friendly (Travel-Required) | San Francisco, CA | New York City, NY About Anthropic Anthropic’s mission...  ...use to monitor model health during training, improving signal-to-noise, reducing latency, and making regressions... 
    Full time
    Work at office
    Remote work
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    3 days ago
  •  ...responsible for training and improving pre-trained models to be deployed into...  ...behavior through human data, evaluations, reward models, and post-training...  ...at the intersection of research, product, and model design...  ...the Role As a Research Engineer / Scientist, you will... 
    Full time
    Work at office
    Relocation package

    openai

    San Francisco, CA
    9 days ago
  • TypeSafe AI is seeking a safety researcher to examine model behavior and help evaluate safety concerns within the team. You’ll build evals, document findings, and contribute to responsible research practices. During the six-week period, you’ll set up access, meet the team... 

    TypeSafe AI

    San Francisco, CA
    5 days ago
  • Zyphra Technologies Inc. in San Francisco is seeking a Research Engineer - Model Architectures to drive design and testing of new model architectures and training methods. You will work closely with the pre-training team to translate research insights into next-generation... 

    Zyphra Technologies Inc.

    San Francisco, CA
    4 days ago
  •  ...training and deploying frontier models for developers and...  ...customers. Cohere is a team of researchers, engineers, designers, and more, who are...  ...the future! Why this role?Evaluation is critical to making...  ...data synthesis pipelines; and improving evaluation efficiency. Build... 
    Full time
    Work at office
    Remote work
    Flexible hours

    cohere

    San Francisco, CA
    3 days ago
  •  ...We are building a frontier AI research company and training our own models end-to-end. The Role We are looking for a research engineer to build the evaluation infrastructure that tells us whether...  ...AI system is actually improving, we'd love to hear from you.... 
    Full time

    mirendil

    San Francisco, CA
    9 days ago
  • Research Engineer — World Model Pretraining(世界モデルの事前学習) 募集概要 Highlanders は、ヒューマノイドロボットと、フィジカルAI(現実の世界で体を動かすAI)の基盤モデル、そしてその学習データを生み出すロボットの群れを、一つの流れとして開発しています。実機が現場で働くほどデータが集まり、モデルが賢くなり、ロボットがさらに働けるようになる。このデータの循環を、日本の工場の現場とサンフランシスコの両方から回しています。 大量の映像・ロボットデータを使った世界モデルの事前学習を... 

    株式会社Highlanders

    San Francisco, CA
    3 days ago
  •  ...based in San Francisco, California. The Role: As a Research Engineer - Language Model Pre-Training , you'll shape our language model roadmap...  ...stack Dataset collection, processing, and evaluation Architecture and methodology research, including optimizer... 
    Full time
    Work at office
    Relocation package

    zyphra

    San Francisco, CA
    9 days ago
  • $170k - $250k

     ...-contributor seat on a small research team. You run the experiments and you build the data and evaluation systems they run on. You do that...  ..., or controlling language models, and running the experiments...  ...explain its assumptions — then improve it. You want your... 
    Full time

    EON Systems

    San Francisco, CA
    3 days ago
  •  ...The role Gauntlet is looking for an AI Research Engineer to help improve how we build, evaluate and use AI agents across the business. We already have agents...  ...produce inconsistent results, and changes to models or tooling can introduce unexpected problems. Today... 
    Work at office
    Remote work
    Work from home

    Remote Services Inc.

    San Francisco, CA
    3 days ago
  •  ...Research EngineerDatacurve provides the frontier coding...  ...world's most advanced models. We absorb and...  ...first autonomous data engine, allowing us to teach...  ...will design, build, and improve the infrastructure we...  ...Strong instincts for evaluation design, benchmark creation... 
    Shift work

    Datacurve

    San Francisco, CA
    3 days ago
  •  ...Research EngineerSuperAnnotate helps the world's leading...  ..., next-generation models powered by high-quality...  ...field. As a Research Engineer, you'll take a research...  ...process to reproduce and improve on it internally.You'...  ...including annotating, evaluating, or sourcing data.Turn... 
    Full time

    SuperAnnotate AI

    San Francisco, CA
    1 day ago
  •  ...ElevenlabsElevenLabs is an AI research and product company...  ...human-like AI voice model. Today, we serve...  .... We are researchers, engineers, and operators. IOI medalists...  ..., architectural improvements, and novel training paradigms...  ...benchmarks and evaluations that tell us whether a... 
    Immediate start
    Remote work

    Eleven Labs

    San Francisco, CA
    5 days ago
  •  ...includes ex-founders and engineers who have built and scaled...  ...We are hiring a Founding Research Engineer to design, prototype...  ...engineering: dataset curation, model training and evaluation, retrieval and tool use,...  ..., and see your work improve care for real people at scale... 

    Lotus

    San Francisco, CA
    1 day ago
  • $275k

     ...Research EngineerMagic's mission is to build safe AGI that accelerates humanity...  ...research and code generation to improve models and solve alignment more reliably...  ...achieve this goal.As a Research Engineer, you'll work on training, evaluating, and serving large AI models and... 
    Relocation

    Magic Inc

    San Francisco, CA
    1 day ago
  •  ...Research Engineer Outset invented a better way to do research: AI-powered user...  ...program to measure and improve how faithfully these systems reproduce...  ...systems built on large language models. Responsibilities Own the evaluation infrastructure that tests... 

    Outset

    San Francisco, CA
    5 days ago
  • $200k - $400k

     ...Research Engineer Decagon is the leading conversational AI platform...  ...-leading conversational AI models that power Decagon's agent,...  ...frontier approaches for training, evaluation, and orchestration across...  ...work end-to-end, ship real improvements, and are trusted to make... 
    Full time
    Work at office
    Local area

    Decagon

    San Francisco, CA
    2 days ago
  •  ...About the Role As a Research Engineer, you’ll be responsible for...  ...industry-leading healthcare AI models and model pipelines that power...  ...work end-to-end, ship real improvements, and are trusted to make...  ...by our customers Design evaluation protocols and benchmarks that... 
    Work at office
    Flexible hours

    Assort Health

    San Francisco, CA
    3 days ago
  •  ...become a massive tax of engineering velocity. Resolve AI...  ...end-to-end, balancing research and engineering to create...  ...production-ready AI models Build and optimize...  ...data for training and evaluation Design and execute...  ...to evaluate AI models, improve performance, and reduce... 
    Work at office
    Visa sponsorship
    Flexible hours

    Resolve AI

    San Francisco, CA
    1 day ago
  •  ...Research Engineer New York - Hybrid; San Francisco Bay Area - Hybrid...  ...is ready for AI. It adapts models to each business and adds human...  ...same approach we have used to improve models for more than 80% of...  ...research organization, focused on evaluation methodology, benchmarks, and... 
    Full time
    Work at office
    Local area
    Remote work

    Invisible Technologies Inc. Defunct

    San Francisco, CA
    5 days ago
  •  ...technology that directly improves patient outcomes. This...  ...daily U.S. X-rays. Our research team brings experience from...  ...for an exceptional Research Engineer or Research Scientist to...  ...advance how we curate data and evaluate frontier AI models for medical imaging.... 
    Full time

    epsilon-health

    San Francisco, CA
    9 days ago
  • $122k - $215k

     ...Research Engineer Waabi, founded by AI visionary Raquel Urtasun, is the leader...  .... You will... Prototype, evaluate, and iterate on solutions,...  ...world data and simulations, to improve the accuracy, robustness,...  ...forecasting, planning, traffic modeling and sensor simulation,... 
    Full time
    Work at office
    Work from home
    Flexible hours

    Waabi

    San Francisco, CA
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Research Engineer, Model Evaluation and Improvement. Be the first to apply!