Senior Research Scientist, Model Evaluation
cohere
Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future! Why this role?Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, you will:
- Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
- Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
- Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.
- Build scalable and reusable tools for digging into model performance.
- You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
- You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
- You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
- You have strong software engineering skills.
- ? An open and inclusive culture and work environment
- ?? Work closely with a team on the cutting edge of AI research
- ? Weekly lunch stipend, in-office lunches & snacks
- ? Full health and dental benefits, including a separate budget to take care of your mental health
- ? 100% Parental Leave top-up for up to 6 months
- ? Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
- ? Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
- ✈️ 6 weeks of vacation (30 working days!)
- Contra Labs, based in San Francisco, seeks a researcher to own research engagements from scoping to delivery, turning ambiguous client... ...and proposals. You will work with cross-disciplinary teams on evaluation, data methodology, and insight synthesis, producing client-ready...Senior
- ...Responsibilities Design and build evaluation suites for manufacturing and... .... Train and fine-tune models using evaluation results to... ...Help establish the team’s research and evaluation practices.... ...of experience operating at a senior or staff level. Hands-on experience...SeniorFull time
- TypeSafe AI is seeking a safety researcher to examine model behavior and help evaluate safety concerns within the team. You’ll build evals, document findings, and contribute to responsible research practices. During the six-week period, you’ll set up access, meet the team...Suggested
$77k - $202k
...Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate, you will provide strategic... ...competitive analysis and market research to inform strategic planning- Utilizing... ...closely with team members. We evaluate these factors thoughtfully to establish...SeniorFull timeH1b- ...of all daily U.S. X-rays. Our research team brings experience from... ...looking for an exceptional Research Scientist to push the frontier of vision-language models for medical imaging. You'll own... ...-scale distributed training, evaluation, and handoff to post-training....SuggestedFull time
- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
$200k - $320k
...Senior Research Scientist EliseAI is seeking world-class experts in Machine Learning and related... ...machine learning and large language models, develop predictive models to enhance... ...experiments, build prototypes, run large-scale evaluations, and drive the project through to...SeniorWork at officeRelocation- ...never been harder or more important. You'll research how to detect fraud and verify... ...and manipulated images, understanding how models perform across populations, and finding... ...images and emerging fraud attacks. Evaluate how reliably our models perform across different...SeniorFull timeFor contractorsInternship
- ...developing next-generation multimodal AI models and a proprietary, high-efficiency... ...role Sciforium is seeking an exceptional Senior Research Scientist specializing in advanced AI/ML to push... ...architectures. Design, implement, and evaluate novel ML algorithms using frameworks...SeniorFlexible hours
- ...problem definition through experimentation, evaluation, deployment, and iteration. Translate... ...clear ML objectives, data strategies, model approaches, and success criteria. Build... ...shift, and workflow disruption. Communicate research findings and technical decisions clearly...SeniorFull timeRemote workFlexible hoursShift work
- Reflection, a research lab, seeks to advance open and accessible intelligence. You will conduct... ...rigorous comparative analyses, build evaluation systems, and develop frameworks that... ...applied teams to translate insights into model improvements. The role emphasizes high agency...
$134.5k - $265.1k
...an experienced SeniorQuantum Research Scientist -- Fault Tolerant Quantum... ...quantum computing offering. As a Senior Research Scientist within... ...software and research tools.Evaluate quantum approaches against relevant... ...-body simulation, financial modelling and risk, or optimization....SeniorLocal area$136.44k - $166.75k
...AI platform for biotech R&D. Scientists use Benchling to design... ...data, and run AI agents and models directly in their workflows.... ....You’ll build the datasets, evaluations, and systems that help close... ...engineers, scientists, and external research partners.Desire to work in a...Work at officeLocal areaMonday to FridayShift work$250k - $300k
...other, come build with us at Crusoe.About This Role:The Senior Staff Software Engineer for the AI Model Lifecycle team will play a crucial role in building... ..., and experiment management: versioning, lineage, evaluation, and reproducible fine-tuning at scale.What You’ll...SeniorTemporary work- ...company at the forefront of advancing large language models (LLMs) and autonomous AI agents. The company develops sophisticated datasets, evaluation methodologies, and data infrastructure that help leading AI research laboratories improve model reasoning, behavior, and...Immediate start
$165.6k - $207k
...provide high quality data and accelerate progress in GenAI research. We are looking for Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation. This role is on the evaluation pod within the GenAI Research...Full time$290.4k - $363k
...that power the most advanced models in the world. Our teams operate... ...intersection of cutting-edge research, large-scale engineering, and... ...the foundational research, evaluation methodologies, and agent/RL infrastructure... ..., and deployed.As a Research Scientist Manager, you will lead a...SeniorFull time$150k - $200k
...creative preference datasets that power benchmarking, evaluation, and post-training for the world's leading AI models and applications. Our work helps define what "... ...this role exists Contra Labs is expanding its research work with frontier AI labs across evaluation, post...- ...when those simulations can be trusted raises difficult questions about measurement, evidence, and generalization. As a Research Scientist in Evaluations, you will own an ambitious, open-ended research agenda with substantial uncertainty. The questions, methods, and path...
- Tenera, Inc. seeks a Research Scientist in Evaluations to lead an ambitious, open-ended research program with substantial uncertainty. You will define questions, develop original approaches, and test ideas with scientific rigor, enjoying autonomy for independent work. You...
- ...We are hiring a commercially minded, operationally pragmatic Senior Manager to define how driverless trucking works as a scalable commercial... ...own the design and evolution of Waabi’s driverless operating model across the full Life of a Load, with a particular focus on our...SeniorFull timeRemote work
$162k - $175.7k
...Senior Scientist / Principal Scientist Induced Proximity Help shape Nurixs... ...induced proximity research strategy. This individual will... ...stabilizers and related mechanisms) Evaluate and integrate new and... ...rational design and predictive modeling of ternary complex formation...SeniorFull timeLocal area$55 - $75 per hour
...About the opportunity Help evaluate AI-generated answers on cybersecurity tasks. You’ll read code, apply the assigned policy to each... ...cybersecurity fundamentals, the ability to read code and spot mistakes in model output, and clear written reasoning. Experience with data...SeniorHourly payFull timeFor contractorsRemote work- ...efficacy, and safety data to make clear next-study and candidate decisions. Responsibilities Design, run, and interpret mouse studies evaluating therapeutic candidates across efficacy, pharmacology, and safety. Select appropriate dose, route, schedule, tissues, timepoints,...SeniorH1bVisa sponsorship
- ...training and improving pre-trained models to be deployed into ChatGPT,... ...behavior through human data, evaluations, reward models, and post-... ...sits at the intersection of research, product, and model design. We... ...As a Research Engineer / Scientist, you will research and develop...Full timeWork at officeRelocation package
$77k - $202k
...Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate The Opportunity As a Strategy& - Strategy Consulting Business... ...objectives Conducting competitive analysis and market research to inform strategic planning Utilizing data analytics to...Senior$204k - $216k
...from and reasons over. You design the data models, schemas, and representations that... ...communities, without fragmenting. The Senior AI/ML Data Model Architect owns that foundation... ...graph, applied AI, infrastructure, and research on shared representations. Align teams...SeniorLocal area$192.2k - $260k
...Sr. Applied Scientist Amazon is seeking an exceptional Sr. Applied... ...insufficient. You will lead research that translates cutting-edge... ...modalities Develop deep learning models for object detection,... ...calibration to model training, evaluation, and real-time deployment...SeniorLocal areaFlexible hoursNight shift- Snorkel AI in New York, NY seeks a Staff or Senior Research Scientist to lead the development of frontier benchmarks and datasets. You will collaborate across research, data operations, product, and GTM to design, implement, and scale high-impact data solutions for real...Senior
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Research Scientist, Model Evaluation. Be the first to apply!
- scientist 1 San Francisco, CA
- research scientist machine learning deep learning San Francisco, CA
- scientist ii San Francisco, CA
- scientist San Francisco, CA
- quality control scientist San Francisco, CA
- entry level research scientist San Francisco, CA
- molecular biology scientist San Francisco, CA
- pharmaceutical scientist San Francisco, CA
- scientist antibody discovery San Francisco, CA
- research scientist - biology San Francisco, CA






