Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Senior Research Scientist, Model Evaluation

Full-time

cohere

Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future! Why this role?Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, you will:

  • Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
  • Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
  • Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.
  • Build scalable and reusable tools for digging into model performance.
You may be a good fit if:
  • You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
  • You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
  • You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
  • You have strong software engineering skills.
If some of the above doesn’t line up perfectly with your experience, we still encourage you to apply! We value and celebrate diversity and strive to create an inclusive work environment for all. We welcome applicants from all backgrounds and are committed to providing equal opportunities. Should you require any accommodations during the recruitment process, please submit an [Accommodations Request Form]( and we will work together to meet your needs. Full-Time Employees at Cohere enjoy these Perks:
  • ? An open and inclusive culture and work environment
  • ?‍? Work closely with a team on the cutting edge of AI research
  • ? Weekly lunch stipend, in-office lunches & snacks
  • ? Full health and dental benefits, including a separate budget to take care of your mental health
  • ? 100% Parental Leave top-up for up to 6 months
  • ? Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
  • ? Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
  • ✈️ 6 weeks of vacation (30 working days!)
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Senior Research Scientist, Model Evaluation in Seattle, WA vacancy
  • $192k - $304.75k

     ...We are now looking for a Research Scientist with a focus in System Software and I/O! NVIDIA is seeking Research Scientists with a focus...  ...research, software infrastructure development and evaluation. ~ A track record of well-documented open-source software... 
    Senior
    Full time
    Work experience placement

    NVIDIA

    Seattle, WA
    4 days ago
  •  ...Senior Java / Spring Boot Technical Architect Bellevue, WA (Onsite – Hybrid Model) Long Term Contract Job Description : The Group Technical Architect is responsible...  ...solutions that meet business needs. Evaluate and recommend tools, technologies, and... 
    Senior
    Long term contract
    Contract work

    Yochana

    Bellevue, WA
    4 days ago
  • $77k - $202k

     ...Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate, you will provide strategic...  ...competitive analysis and market research to inform strategic planning- Utilizing...  ...closely with team members. We evaluate these factors thoughtfully to establish... 
    Senior
    Full time
    H1b

    PwC

    Seattle, WA
    3 days ago
  •  ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other...  ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value... 
    Senior
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    Seattle, WA
    1 day ago
  • $168.1k - $227.4k

    As a Senior Forward Deployed Engineer (Senior FDE) on the Custom Model and Optimization team, you are the technical lead embedded...  ...success metrics and the evaluation bar, and set technical roadmap...  ...leadership to a team of FDEs and scientists on the engagement, breaking ambiguous... 
    Senior
    Internship
    Local area
    Flexible hours

    AmazonWebServices

    Seattle, WA
    16 hours ago
  •  ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other...  ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value... 
    Senior
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    Seattle, WA
    1 day ago
  • $159.75k - $255.6k

     ...are seeking a skilled and innovative Senior AI Research Scientist to join a new team focusing on agentic...  ...and cross-functional teams to train models and develop cutting-edge algorithms and...  ...strategy through model development, evaluation, deployment, and iteration in... 
    Senior
    Work experience placement
    Work at office
    Remote work

    Axon

    Seattle, WA
    8 days ago
  • $216.2k - $394k

     ...add something!DescriptionAs a Senior Staff/Principal Engineer on the Foundation Model Compute Infrastructure team, you...  ...foundation model training, fine-tuning, evaluation, and inference.You will tackle...  ...tools and services that enable researchers to launch, monitor, debug, and... 
    Senior
    Relocation

    Apple

    Seattle, WA
    2 days ago
  • $147k - $220k

     ...Senior Research Engineer Seattle, Washington | Hybrid The position is a Senior Research Engineer on the model scaling team, reporting to a Director of AI Research. The work covers model training and the supporting infrastructure. Training runs use approximately 6... 
    Senior
    Remote work
    Visa sponsorship

    Intelix.AI

    Seattle, WA
    3 days ago
  • $192k - $304.75k

    We are now looking for a Senior Robotics Research Scientist! NVIDIA is at the forefront of the AI and robotics...  ...the Cosmos and GR00T-N foundation models, as well as the Newton physics...  ...robust workflows for model training and evaluation.Ways to stand out from the crowd:Prior... 
    Senior
    Full time
    Work at office

    Nvidia

    Seattle, WA
    16 hours ago
  • $184.7k - $324.8k

     ...an Applied Machine Learning Research Scientist to join our Accessibility...  ...Intelligence (HCMI) group advances ML modeling and interactive...  ...ML, and synthetic data and evaluation methodologies.Minimum QualificationsPhD...  ...record commensurate with seniority.Proven history of turning... 
    Senior
    Relocation

    Apple

    Seattle, WA
    3 days ago
  • $142.3k - $263.3k

     ...rely on human-in-the-loop evaluation, the quality of our products...  ...Operations team is looking for a Senior Applied Scientist to join our growing team....  ...demands fluency across research thinking and engineering execution...  ...view on when not to use a model or agent is as valued here... 
    Senior
    Relocation

    Apple

    Seattle, WA
    3 days ago
  • $77k - $202k

     ...Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate The Opportunity As a Strategy& - Strategy Consulting Business...  ...objectives Conducting competitive analysis and market research to inform strategic planning Utilizing data analytics to... 
    Senior

    PwC (US)

    Seattle, WA
    16 hours ago
  •  ...responsible for building machine learning models and systems to protect our users from...  ...paradigm for this direction and produce research outcomes with significant industry influence...  ...similar technologies.- Experience with evaluation of AI systems, LLM application & agent... 
    Flexible hours
    Shift work

    TikTok

    Seattle, WA
    2 days ago
  • $102k - $112k

     ...Wellness programs, commuter benefits, and a vibrant company culture Join Howard University and thrive with us! BASIC FUNCTION: Senior Research Scientists positions are available at Howard University to work on an ONR-supported research project led by Dr. Gloria Washington to... 
    Senior
    Flexible hours

    Howard University

    Seattle, WA
    3 days ago
  •  ...Senior Merchant Mariner Evaluation Specialist (Engine) This vacancy is for a GS-2101-13, Senior Merchant Mariner Evaluation Specialist (Engine) located in the Department of Homeland Security, U.S. Coast Guard, NMC MARINER EXAMINATIONS BRANCH in location to be determined... 
    Senior

    US Government Jobs

    Seattle, WA
    2 days ago
  • $136k

     ...passionate, talented, and inventive Senior Applied Scientist with a strong machine...  ...AI, Large Language Model (LLM), Natural Language Understanding...  ..., Responsible AI, Agent, Evaluation, and Model Adaptation. As...  ...contribute to the wider research community. You will gain hands... 
    Senior

    Amazon

    Seattle, WA
    16 hours ago
  •  ...iteratively improves itself through feedback, evaluation, and automated learning loops. You will...  ...evaluation harnesses and metrics before models - because a self-improving system is...  ...go non-stationary. Collaborate with research and product to turn methods into robust,... 
    Senior

    Hippocratic AI

    Seattle, WA
    2 days ago
  • $167.44k - $226.07k

     ...control, planning, or evaluation ~ Take problems from...  ...requirements through clear research questions, hypotheses,...  ...engineers to take models into production grade...  ...product managers, and senior leaders Preferred...  ...mentoring more junior scientists and engineers Experience... 
    Senior
    Local area

    Siemens AG

    Bellevue, WA
    3 days ago
  •  ...grid. As a Power System Intern focused on foundational models, you will contribute to research and prototyping efforts that combine power system expertise...  ...model performance on utility-specific tasks. Evaluate model behavior, accuracy, and domain adaptation. Support... 
    Permanent employment
    Summer work
    Internship
    Relocation package

    GE Vernova

    Bellevue, WA
    3 days ago
  • $205.4k - $308.5k

     ...as the methods used to evaluate them. At Apple, where AI...  ...alongside measurement scientists (psychometrics, validity theory), ML researchers, and platform engineers...  ...preference learning, reward modeling, or calibration theory,...  ...(early-career to senior researchers). What unites... 
    Relocation

    Apple

    Seattle, WA
    3 days ago
  •  ...HDR, our employee‑ownership model shapes everything we do,...  ...the world. In the role of Senior Environmental Scientist, we'll count on you to: •...  ...wildlife surveys, and habitat evaluations while managing task level...  ..., problem-solving and research abilities Ability to work... 
    Senior
    Full time
    Local area

    HDR

    Bellevue, WA
    1 day ago
  • $159.75k - $255.6k

     ...innovative Machine Learning Scientists to join our AI team, focusing...  ...Robotics. As a key member of our research and development efforts, you...  ...of-the-art in Large Language Models (LLMs), Multimodal Large...  ...hardware architectures. Evaluate the performance of LLM, MLLM,... 
    Senior
    Work experience placement
    Work at office
    Remote work

    Axon

    Seattle, WA
    4 days ago
  • $152.2k - $205.9k

     ...That's the question the Price Perception Research team answers at scale, for millions of customers worldwide. We're seeking a Senior UX Researcher to lead CX research into how...  ..., interviews, diary studies, concept evaluation, and survey design, choosing the right method... 
    Senior
    Work experience placement
    Remote work
    Worldwide
    Flexible hours

    Amazon

    Seattle, WA
    16 hours ago
  •  ...shape how population-level health data is understood, interpreted, and acted upon? We're looking for a Senior Epidemiologist to evaluate complex epidemiological models, dissect study designs, and surface the insights that drive real public health decisions. Senior... 
    Senior
    Hourly pay
    Ongoing contract
    Contract work
    Freelance
    Remote work
    Flexible hours

    Alignerr Corp.

    Seattle, WA
    2 days ago
  • $119.8k - $234.7k

     ...division. We are looking for a Senior Economist to join our team....  ...we can use less expensive AI models for the same business...  ...statistics predictive analytics, research) OR Master's Degree in...  ...formulating research questions and evaluating relevant literature. ~ Doctorate... 
    Senior
    Ongoing contract
    Local area

    Microsoft Corporation

    Seattle, WA
    16 hours ago
  • $136.03k - $164.84k

     ...Cell Therapy Thematic Research Center is one of BMS’s...  ...discovery and translational scientists are focused on...  ...Responsibilities: In the role as a Senior Scientist within the...  ...related to the evaluation of lentiviral vectors...  ...across four work models: site-essential, site-... 
    Senior
    Hourly pay
    Full time
    Summer work
    Live in
    Local area
    Remote work
    Flexible hours

    Bristol Myers Squibb

    Seattle, WA
    12 days ago
  • $250k - $350k

     ...learning environments, and frontier research benchmarks that improve model capabilities in software...  ...deployment signals into better data, evaluations, and more capable models. Learn more...  ...is seeking exceptional  Research Scientists to join our STEM research organization... 
    Senior

    Turing

    Seattle, WA
    24 days ago
  • $120k - $160k

     ...in biotech and healthcare by providing researchers and companies with the building blocks...  ...0 countries and regions. Position: Senior Scientist / Research Scientist, Molecular...  ...and service offerings. Critically evaluate experimental data, draw actionable conclusions... 
    Senior
    Temporary work
    Flexible hours
    Shift work

    GenScript

    Seattle, WA
    13 days ago
  • $110k - $220k

     ...client management role, business development, ability to handle client inquiries on a day-to-day basis, professional attitude, able to evaluate issues, objections, operational and plan design issues; excellent personal, telephone and communication skills, etc. Experience... 
    Senior
    Minimum wage
    Work at office
    Local area
    Remote work
    Flexible hours
    3 days per week
    1 day per week

    Guy Carpenter

    Seattle, WA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Senior Research Scientist, Model Evaluation. Be the first to apply!