Senior Research Scientist, Model Evaluation
cohere
Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future! Why this role?Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, you will:
- Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
- Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
- Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.
- Build scalable and reusable tools for digging into model performance.
- You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
- You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
- You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
- You have strong software engineering skills.
- ? An open and inclusive culture and work environment
- ?? Work closely with a team on the cutting edge of AI research
- ? Weekly lunch stipend, in-office lunches & snacks
- ? Full health and dental benefits, including a separate budget to take care of your mental health
- ? 100% Parental Leave top-up for up to 6 months
- ? Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
- ? Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
- ✈️ 6 weeks of vacation (30 working days!)
$192k - $304.75k
...We are now looking for a Research Scientist with a focus in System Software and I/O! NVIDIA is seeking Research Scientists with a focus... ...research, software infrastructure development and evaluation. ~ A track record of well-documented open-source software...SeniorFull timeWork experience placement- ...Senior Java / Spring Boot Technical Architect Bellevue, WA (Onsite – Hybrid Model) Long Term Contract Job Description : The Group Technical Architect is responsible... ...solutions that meet business needs. Evaluate and recommend tools, technologies, and...SeniorLong term contractContract work
$77k - $202k
...Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate, you will provide strategic... ...competitive analysis and market research to inform strategic planning- Utilizing... ...closely with team members. We evaluate these factors thoughtfully to establish...SeniorFull timeH1b- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
$168.1k - $227.4k
As a Senior Forward Deployed Engineer (Senior FDE) on the Custom Model and Optimization team, you are the technical lead embedded... ...success metrics and the evaluation bar, and set technical roadmap... ...leadership to a team of FDEs and scientists on the engagement, breaking ambiguous...SeniorInternshipLocal areaFlexible hours- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
$159.75k - $255.6k
...are seeking a skilled and innovative Senior AI Research Scientist to join a new team focusing on agentic... ...and cross-functional teams to train models and develop cutting-edge algorithms and... ...strategy through model development, evaluation, deployment, and iteration in...SeniorWork experience placementWork at officeRemote work$216.2k - $394k
...add something!DescriptionAs a Senior Staff/Principal Engineer on the Foundation Model Compute Infrastructure team, you... ...foundation model training, fine-tuning, evaluation, and inference.You will tackle... ...tools and services that enable researchers to launch, monitor, debug, and...SeniorRelocation$147k - $220k
...Senior Research Engineer Seattle, Washington | Hybrid The position is a Senior Research Engineer on the model scaling team, reporting to a Director of AI Research. The work covers model training and the supporting infrastructure. Training runs use approximately 6...SeniorRemote workVisa sponsorship$192k - $304.75k
We are now looking for a Senior Robotics Research Scientist! NVIDIA is at the forefront of the AI and robotics... ...the Cosmos and GR00T-N foundation models, as well as the Newton physics... ...robust workflows for model training and evaluation.Ways to stand out from the crowd:Prior...SeniorFull timeWork at office$184.7k - $324.8k
...an Applied Machine Learning Research Scientist to join our Accessibility... ...Intelligence (HCMI) group advances ML modeling and interactive... ...ML, and synthetic data and evaluation methodologies.Minimum QualificationsPhD... ...record commensurate with seniority.Proven history of turning...SeniorRelocation$142.3k - $263.3k
...rely on human-in-the-loop evaluation, the quality of our products... ...Operations team is looking for a Senior Applied Scientist to join our growing team.... ...demands fluency across research thinking and engineering execution... ...view on when not to use a model or agent is as valued here...SeniorRelocation$77k - $202k
...Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate The Opportunity As a Strategy& - Strategy Consulting Business... ...objectives Conducting competitive analysis and market research to inform strategic planning Utilizing data analytics to...Senior- ...responsible for building machine learning models and systems to protect our users from... ...paradigm for this direction and produce research outcomes with significant industry influence... ...similar technologies.- Experience with evaluation of AI systems, LLM application & agent...Flexible hoursShift work
$102k - $112k
...Wellness programs, commuter benefits, and a vibrant company culture Join Howard University and thrive with us! BASIC FUNCTION: Senior Research Scientists positions are available at Howard University to work on an ONR-supported research project led by Dr. Gloria Washington to...SeniorFlexible hours- ...Senior Merchant Mariner Evaluation Specialist (Engine) This vacancy is for a GS-2101-13, Senior Merchant Mariner Evaluation Specialist (Engine) located in the Department of Homeland Security, U.S. Coast Guard, NMC MARINER EXAMINATIONS BRANCH in location to be determined...Senior
$136k
...passionate, talented, and inventive Senior Applied Scientist with a strong machine... ...AI, Large Language Model (LLM), Natural Language Understanding... ..., Responsible AI, Agent, Evaluation, and Model Adaptation. As... ...contribute to the wider research community. You will gain hands...Senior- ...iteratively improves itself through feedback, evaluation, and automated learning loops. You will... ...evaluation harnesses and metrics before models - because a self-improving system is... ...go non-stationary. Collaborate with research and product to turn methods into robust,...Senior
$167.44k - $226.07k
...control, planning, or evaluation ~ Take problems from... ...requirements through clear research questions, hypotheses,... ...engineers to take models into production grade... ...product managers, and senior leaders Preferred... ...mentoring more junior scientists and engineers Experience...SeniorLocal area- ...grid. As a Power System Intern focused on foundational models, you will contribute to research and prototyping efforts that combine power system expertise... ...model performance on utility-specific tasks. Evaluate model behavior, accuracy, and domain adaptation. Support...Permanent employmentSummer workInternshipRelocation package
$205.4k - $308.5k
...as the methods used to evaluate them. At Apple, where AI... ...alongside measurement scientists (psychometrics, validity theory), ML researchers, and platform engineers... ...preference learning, reward modeling, or calibration theory,... ...(early-career to senior researchers). What unites...Relocation- ...HDR, our employee‑ownership model shapes everything we do,... ...the world. In the role of Senior Environmental Scientist, we'll count on you to: •... ...wildlife surveys, and habitat evaluations while managing task level... ..., problem-solving and research abilities Ability to work...SeniorFull timeLocal area
$159.75k - $255.6k
...innovative Machine Learning Scientists to join our AI team, focusing... ...Robotics. As a key member of our research and development efforts, you... ...of-the-art in Large Language Models (LLMs), Multimodal Large... ...hardware architectures. Evaluate the performance of LLM, MLLM,...SeniorWork experience placementWork at officeRemote work$152.2k - $205.9k
...That's the question the Price Perception Research team answers at scale, for millions of customers worldwide. We're seeking a Senior UX Researcher to lead CX research into how... ..., interviews, diary studies, concept evaluation, and survey design, choosing the right method...SeniorWork experience placementRemote workWorldwideFlexible hours- ...shape how population-level health data is understood, interpreted, and acted upon? We're looking for a Senior Epidemiologist to evaluate complex epidemiological models, dissect study designs, and surface the insights that drive real public health decisions. Senior...SeniorHourly payOngoing contractContract workFreelanceRemote workFlexible hours
$119.8k - $234.7k
...division. We are looking for a Senior Economist to join our team.... ...we can use less expensive AI models for the same business... ...statistics predictive analytics, research) OR Master's Degree in... ...formulating research questions and evaluating relevant literature. ~ Doctorate...SeniorOngoing contractLocal area$136.03k - $164.84k
...Cell Therapy Thematic Research Center is one of BMS’s... ...discovery and translational scientists are focused on... ...Responsibilities: In the role as a Senior Scientist within the... ...related to the evaluation of lentiviral vectors... ...across four work models: site-essential, site-...SeniorHourly payFull timeSummer workLive inLocal areaRemote workFlexible hours$250k - $350k
...learning environments, and frontier research benchmarks that improve model capabilities in software... ...deployment signals into better data, evaluations, and more capable models. Learn more... ...is seeking exceptional Research Scientists to join our STEM research organization...Senior$120k - $160k
...in biotech and healthcare by providing researchers and companies with the building blocks... ...0 countries and regions. Position: Senior Scientist / Research Scientist, Molecular... ...and service offerings. Critically evaluate experimental data, draw actionable conclusions...SeniorTemporary workFlexible hoursShift work$110k - $220k
...client management role, business development, ability to handle client inquiries on a day-to-day basis, professional attitude, able to evaluate issues, objections, operational and plan design issues; excellent personal, telephone and communication skills, etc. Experience...SeniorMinimum wageWork at officeLocal areaRemote workFlexible hours3 days per week1 day per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Research Scientist, Model Evaluation. Be the first to apply!
- scientist 1 Seattle, WA
- research scientist machine learning deep learning Seattle, WA
- scientist ii Seattle, WA
- senior research scientist - machine learning Seattle, WA
- scientist Seattle, WA
- quality control scientist Seattle, WA
- molecular biology scientist Seattle, WA
- pharmaceutical scientist Seattle, WA
- research scientist - biology Seattle, WA
- cell culture scientist Seattle, WA






