Senior Research Scientist, Model Evaluation
cohere
Who are we?Our mission is to scale intelligence to serve humanity. We’re training and deploying frontier models for developers and enterprises who are building AI systems to power magical experiences like content generation, semantic search, RAG, and agents. We believe that our work is instrumental to the widespread adoption of AI. We obsess over what we build. Each one of us is responsible for contributing to increasing the capabilities of our models and the value they drive for our customers. We like to work hard and move fast to do what’s best for our customers. Cohere is a team of researchers, engineers, designers, and more, who are passionate about their craft. Each person is one of the best in the world at what they do. We believe that a diverse range of perspectives is a requirement for building great products. Join us on our mission and shape the future! Why this role?Evaluation is critical to making progress in scaling intelligence. As models continue to become superhuman in many real-world use cases, we must continue to develop new evaluation techniques that accurately reflect what models are already capable of, as well as set the agenda for what future models should be capable of. In this role, you are responsible for creating these next-generation evaluation methods and infrastructure to measure LLM progress. As a Senior Research Scientist, Model Evaluation, you will:
- Create ambitious new evaluation benchmarks that push the limits of what our models can accomplish.
- Work on highly cross-functional teams to translate model feedback into trustworthy, repeatable evaluations.
- Conduct research to advance the state-of-the-art in LLM evaluation methods, including training LLM judges; refining LLM-based data synthesis pipelines; and improving evaluation efficiency.
- Build scalable and reusable tools for digging into model performance.
- You enjoy rapidly building prototypes that demonstrate the boundaries of what LLMs are capable of, and you have developed resources to measure those capabilities.
- You have spent dozens of hours reviewing complex data and LLM outputs to ensure high data quality.
- You are obsessive about rigorously measuring AI capabilities, and also about making sure your measurements actually align with the capabilities you care about.
- You have strong software engineering skills.
- ? An open and inclusive culture and work environment
- ?? Work closely with a team on the cutting edge of AI research
- ? Weekly lunch stipend, in-office lunches & snacks
- ? Full health and dental benefits, including a separate budget to take care of your mental health
- ? 100% Parental Leave top-up for up to 6 months
- ? Personal enrichment benefits towards arts and culture, fitness and well-being, quality time, and workspace improvement
- ? Remote-flexible, offices in Toronto, New York, San Francisco, London and Paris, as well as a co-working stipend
- ✈️ 6 weeks of vacation (30 working days!)
$295k
...company. We build cutting-edge foundation AI models and end-to-end products that are... ...for our customers. Cohere is a team of researchers, engineers, designers, and more, who are... ...Berlin and Seoul. Join us!Role Overview:Evaluation is critical to making progress in scaling...SeniorFull timeWork at officeLocal areaRemote workHome office$77k - $202k
...Strategy& - Strategy Consulting Business Model Reinvention - Senior Associate, you will provide strategic... ...competitive analysis and market research to inform strategic planning- Utilizing... ...closely with team members. We evaluate these factors thoughtfully to establish...SeniorFull timeH1b- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
- ...accenture.com.We Are:Accenture's Enterprise Operating Model practice. We partner with Boards, CEOs, and other... ...out the best in teams.The Work:Engage with senior clients on their biggest business challenges.Evaluate actionable solutions to these challenges, the value...SeniorWork experience placementLive inWork at officeLocal area
$193.3k - $261.5k
As a Senior Forward Deployed Engineer (Senior FDE) on the Custom Model and Optimization team, you are the technical lead embedded... ...success metrics and the evaluation bar, and set technical roadmap... ...leadership to a team of FDEs and scientists on the engagement, breaking ambiguous...SeniorInternshipLocal areaFlexible hours$190k - $245k
...See yourself at Dataminr As a Senior Research Scientist at Dataminr, you will tackle some of the... ...state of the art in large language models, generative AI, and agentic systems while... ...and adaptation, distillation, evaluation and benchmarking, agentic systems, retrieval...SeniorWork at officeFlexible hours$65k - $120k
...thousands of physicians and scientists who have helped to... ...education, scientific research, and direct patient... ...to join our team as a Senior Research Scientist within... ...-centered data for evaluation of interventions and characteristics... ...to complex modeling to interactive data...SeniorFull time$120k - $140k
... Research -- Basic Science/Technical New York • ID: _... ...thousands of physicians and scientists who have helped to shape the... ...opportunity to join our team as a Senior Research Scientist. The... ...epidemiology, health services research, evaluation, biostatistics, and medical...SeniorFull time$172.8k - $359k
...a Forward Deployed Engineer Senior Manager, you will be joining... ...and interdisciplinary team of scientists and engineers who love to tackle... ...into a prioritized, scalable Model Engineering roadmap that... ...that enable implementation, evaluation, governance, monitoring, and...SeniorImmediate startRemote work$200k - $320k
...strengthen our expansive research team. We are looking... ..., and computer scientists—who are passionate about... ...learning and large language models, develop predictive... ...data, and more. As a Senior Research Scientist you... ...prototypes, run large‑scale evaluations, and drive the project...SeniorWork at officeLocal areaRelocation$135k - $150k
...Position: Senior Research Scientist Department: NYU McSilver Institute for Poverty Policy and Research Supervisor: Dr. Fred Ssewamala... ...highly collaborative research environment. Oversee performance evaluations, professional development, and day-to-day workloads of...SeniorFull timeWork experience placementWork at officeImmediate start3 days per week- ...Description: As a Senior AI Research Scientist, you will bridge cutting-edge AI research with production... ...focus on applying Large Language Models (LLMs), AI Agents, and Multimodal AI... ...through implementation, deployment, evaluation, and continuous iteration. Build AI...SeniorFull time
$174k - $252k
Author research papers to share and generate impact of research results... ..., framework, design, and evaluation metrics for research solution... ...AI accelerators (TPUs, GPUs), model architectures (encoders,... ...other researchers. As a Research Scientist on the Graph Neural Network (...Senior$134.5k - $265.1k
...an experienced SeniorQuantum Research Scientist -- Fault Tolerant Quantum... ...quantum computing offering. As a Senior Research Scientist within... ...software and research tools.Evaluate quantum approaches against relevant... ...-body simulation, financial modelling and risk, or optimization....SeniorLocal area- ...Job Description Job Title: Language Model Evaluator Job Summary We are seeking a Language... ...collaborating with AI Engineers, Data Scientists, Prompt Engineers, and Product teams to... ...evaluation Contributions to AI evaluation research or open-source projects Key...Full timeRemote work
- Mecka AI in New York seeks an experienced Research Scientist to advance large-scale vision-language and multimodal foundation models. You will work on model training, post-training... ...across data, training strategies, evaluation, and optimization, exploring emerging approaches...Senior
$40 - $125 per hour
...Drug Discovery Scientist, AI Trainer (Remote, Flexible Hours)100% Remote | Work When You Want | Part-Time or Full-Time | Freelance... ...discovery scientists to work as remote AI trainers and evaluate how frontier AI models handle real discovery and preclinical work: SAR analysis,...Hourly payFull timeContract workTemporary workPart timeFreelanceRemote workWork from home10 hours per weekFlexible hoursAfternoon shift$60 - $90 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include... ...and Jack Dorsey . Position: Machine Learning Engineer — Model Evaluation & Experimentation Type: Contract Compensation: $60...Contract workSummer workRemote work$127.4k - $236.6k
...Reuters Labs builds and ships custom models for professional workflows... ...more on the latest model release.The research team trains and evaluates the Thomson models to perform at a... ...evaluation efforts.We are hiring scientists at Senior and Lead level across three areas...SeniorFull timeLocal areaFlexible hours$165.6k - $207k
...provide high quality data and accelerate progress in GenAI research. We are looking for Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation. This role is on the evaluation pod within the GenAI Research...Full time$290.4k - $363k
...that power the most advanced models in the world. Our teams operate... ...intersection of cutting-edge research, large-scale engineering, and... ...the foundational research, evaluation methodologies, and agent/RL infrastructure... ..., and deployed.As a Research Scientist Manager, you will lead a...SeniorFull time$150k - $200k
...creative preference datasets that power benchmarking, evaluation, and post-training for the world's leading AI models and applications. Our work helps define what "... ...this role exists Contra Labs is expanding its research work with frontier AI labs across evaluation, post...Work at office$55 - $75 per hour
...About the opportunity Help evaluate AI-generated answers on cybersecurity tasks. You’ll read code, apply the assigned policy to each... ...cybersecurity fundamentals, the ability to read code and spot mistakes in model output, and clear written reasoning. Experience with data...SeniorHourly payFull timeFor contractorsRemote work$262.5k - $299.6k
Applied Researcher 5 (AI Foundations: Evaluation and Agentic Workflows) At Capital One, we are creating trustworthy... ...a cross-functional team of data scientists, software engineers, machine... ...textual data. Build AI foundation models through all phases of development,...Full timePart timeWork at officeLocal areaFlexible hours- ...growth marketers, is seeking a Senior Marketing Scientist to drive the rigor,... ...frameworks—spanning Econometric Modeling (MMM), Multi-Touch... ...curves. Design, execute, and evaluate incrementality frameworks,... ...forefront of marketing science research, continually testing new open...Senior
- ...The Research & Intelligence team builds the machine learning and... ..., temporal and causal modeling, data valuation systems, and... ...data assets. We are seeking a Senior Applied Scientist to lead and expand key research... ...standards for model evaluation, monitoring, and lifecycle...SeniorWork at officeLocal areaRemote work
$167.4k - $310.8k
...development. Roche’s Research and Early Development... ...and novel computational models to drive impact. Seamless... ...(AI) to assist our scientists in both pRED and gRED... ...techniques. We are seeking a Senior or Principal Machine... ...strategies, and evaluation methodologies.Model Capability...SeniorFull timeLocal areaWorldwideRelocation package- ...Senior Applied ScientistAI assistants are only as... ...found. Modern language models run throughout, and we... ...for a Senior Applied Scientist to help drive this work... ...data through modeling, evaluation and deployment, while... ...product, ranking and research teams to take new techniques...Senior
$167.44k - $226.07k
...Senior Applied Scientist Siemens builds the systems the physical world runs... ...intersection of machine learning research, real world data, and... ...experimentation, and into models that run in production. You... ...series, control, planning, or evaluation Take problems from...SeniorLocal area$183.8k - $248.7k
...Applied Scientist We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic... ...datasets aligned with model development needs Build and... ...degree and 6+ years of applied research experience Experience in...SeniorFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior Research Scientist, Model Evaluation. Be the first to apply!
- scientist 1 New York, NY
- research scientist machine learning deep learning New York, NY
- scientist ii New York, NY
- junior research scientist New York, NY
- scientist New York, NY
- quality control scientist New York, NY
- entry level research scientist New York, NY
- molecular biology scientist New York, NY
- pharmaceutical scientist New York, NY
- scientist antibody discovery New York, NY







