Senior LLM Evaluation Scientist
cohere
Cohere is seeking a Senior Research Scientist, Model Evaluation, to create ambitious evaluation benchmarks and scale evaluation infrastructure for enterprise AI. You will work with cross-functional teams to translate model feedback into trustworthy measurements and to push the frontiers of LLM evaluation. The role emphasizes rigorous measurement, practical tooling, and collaboration across research and engineering, with remote-friendly policies and opportunities to influence model evaluation at #J-18808-Ljbffr cohere
Vacancy posted 10 hours ago
Similar jobs that could be interesting for youBased on the Senior LLM Evaluation Scientist in New York, NY vacancy
- ...Montreal, Seoul, Germany and Paris. Join us!Why this role?Evaluation is critical to making progress in scaling... ...generation evaluation methods and infrastructure to measure LLM progress.As a Senior Research Scientist, Model Evaluation, you will:Create ambitious new evaluation...SeniorFull timeWork at officeLocal areaRemote workHome office
$204k - $259k
Senior Research Scientist, Foundation Model (LLM/VLM) Waymo Position type: Full‑time Location: New York Waymo is an autonomous driving technology company... ...Design and run experiments training and evaluating large deep learning models Present results to peers...SeniorFull time- Cohere is seeking a Senior Research Engineer, Model Evaluation, to create next‑generation evaluation methods and scalable infrastructure. You will develop... ..., and you will push the state‑of‑the‑art in LLM evaluation methods while building tools used across technical...Senior
- Omaze is looking for a Senior Applied AI Scientist to join their highly skilled Tech team. This role focuses on AI system evaluation, measurement, and optimization, combining data science... ...will work on evaluation frameworks for LLM features and analyze production impact...Senior
$167.4k - $310.8k
...Intelligence (AI) to assist our scientists in both pRED and gRED to... ...techniques. We are seeking a Senior or Principal Machine Learning... ...architectures, training strategies, and evaluation methodologies.Model... ...leadership and project ownership.LLM Expertise: Extensive...SeniorFull timeLocal areaWorldwideRelocation package- ...business decisions. You'll collaborate with scientists and engineers to deliver solutions while... ..., tree-based, deep neural networks, and LLM-based). You'll develop end-to-end ML... ...few-shot learning, fine-tuning, and model evaluation Production Deployment: Build scalable,...SeniorWorldwide
Amazon
New York, NY1 day ago - ...heterogeneous data assets. We are seeking a Senior Applied Scientist to lead and expand key research... ...Establish rigorous standards for model evaluation, monitoring, and lifecycle management... ...embeddings, information retrieval, and LLM-based systems Proven ability to take research...SeniorWork at officeLocal areaRemote work
Carbon Arc
New York, NY2 days ago - OpenRouter in New York seeks a Research Scientist to advance how the world understands, evaluates, and routes large language models. You will design experiments, build evaluation frameworks, and publish findings that influence rankings and routing decisions. You will collaborate...
$168k - $264.5k
...future of large language models. We are looking for a Senior Scientist to join our team and help advance our capabilities... ...:Build synthetic data generation pipelines using LLM-based methods and automated quality evaluation, producing datasets that improve the pre- and post...SeniorFull timeRemote work$236k - $260k
...Role: We are seeking an exceptional Senior Applied Scientist to join our Applied Science team. In... ...rigor, and build the standards and evaluation tooling the team relies on Build a... ...doctor. Fine-tune and productionize an LLM-based primary care experience on our website...SeniorFlexible hoursGarner Health
New York, NY1 day ago$183.8k - $248.7k
...advertising products.About This Role:As a Senior Scientist for the team, you will have the... ...subject matter expertise in the area of ML, LLM and GenAI models. You will invent new product... ...* Design and conduct A/B experiments to evaluate proposed solutions based on in-depth...SeniorFlexible hours$127.4k - $236.6k
Senior Applied Scientist, Document UnderstandingAbout the RoleThis is an applied science position focused... ...model development, distillation, evaluation, and deployment. You work independently... ...taxonomies and extract rich metadataDevelop LLM-based knowledge graph construction...SeniorFull timeLocal areaFlexible hours- Rex.zone is seeking an AI Research Scientist to lead applied AI research projects for US-based customers, translating open-ended questions into measurable experiments in LLM evaluation and RLHF data design. You will evaluate prompts, design datasets, and work with cross...Remote jobHourly payFlexible hours
$216k - $270k
Scale Labs, Research Scientist — Frontier Risk EvaluationsAs the leading data and evaluation partner for frontier AI companies, Scale plays an integral role in understanding... ...a background in data science roles related to LLM technologies.Experience with red-teaming or...Full time- Scale AI, Inc. is seeking a qualified candidate to tackle sophisticated machine learning problems related to large language models. The ideal applicant will possess hands-on experience with frameworks like Pytorch and have a proven research background in top ML venues. ...Senior
- Waymo is looking for a Senior Research Scientist for its Foundation Model team in New York. In this hybrid role, you will conduct applied research in deep learning to address autonomous driving challenges. The ideal candidate holds a Masters or Ph.D. and has over 3 years...Senior
$143.63k - $299.38k
...and production‑grade execution. You don’t just follow the latest LLM trends; you understand the mechanics of transformer... ...distillation. You believe that a model is only as good as its evaluation framework and its ability to perform in a low‑latency production...SeniorWork at officeFlexible hours$200k - $320k
...theoretical physicists, and computer scientists—who are passionate about... ...large data, and more. As a Senior Research Scientist you will... ...engagement, conversational AI, and LLM safety; turn them into road‑... ...prototypes, run large‑scale evaluations, and drive the project...SeniorWork at officeLocal areaRelocation$122.4k - $170k
About the Senior Product Scientist at Headspace: We are seeking a Senior Product Scientist to own and... ...write SQL and Python, contribute to model evaluation, and be a proficient technical partner... ...learning, contextual bandit, or LLM‑based features for real‑time personalization...SeniorFull timeWork at officeLocal areaRemote work3 days per week- ...Amazon Development Center U.S., Inc. in New York, NY is seeking an Applied Scientist III to design, develop, evaluate and deploy data-driven ML and NL solutions. You will work on statistical models, Bayesian methods and deep learning, building models in production and...SeniorFull timeRemote work
Amazon Science
New York, NY1 day ago $183.8k - $248.7k
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world...SeniorFlexible hours$200k - $240k
...scale. About the role: We are seeking an exceptional Applied Scientist to work on core search systems that power our product, with a... ...Garner selects high-value providers for members by developing the evaluation frameworks, objective functions, and algorithms that drive our...SeniorWork at officeWork visaFlexible hours3 days per weekGarner Health
New York, NY5 days ago- Scale Labs, Research Scientist — Frontier Risk Evaluations As the leading data and evaluation partner for frontier AI companies, Scale plays an integral... ...benchmarks, or a background in data science roles related to LLM technologies. Experience with red‑teaming or adversarial...
$183.8k - $248.7k
...Description We are seeking a Sr. Applied Scientist to develop and optimize Visual Inertial Odometry (VIO) and sensor fusion systems for... ...geometric approaches Build and maintain calibration, evaluation, and benchmarking infrastructure for perception systems Collaborate...SeniorFlexible hoursAmazon Science
New York, NY5 days ago$183.8k - $248.7k
...team is at the forefront of machine learning research. Led by scientists including Dean Foster, the team is dedicated to developing innovative... ...s operations. Key job responsibilities Design, implement, and evaluate innovative models, agents, and software prototypes....SeniorFull timeTemporary workSeasonal workFlexible hoursAmazon
New York, NY1 day ago$192k - $304.75k
...residual histories, discretized operators, meshes, geometry, simulation outputs, performance counters, and physics constraints.Define evaluation methods that measure numerical impact, including convergence rate, failure rate, conservation, memory footprint, correctness, and...SeniorFull timeRemote work$192k - $304.75k
...building software, systems, and research platforms that help scientists and engineers solve problems that were once out of reach. We are... ...-precision algorithms, and sparse direct/iterative hybrids.Evaluate algorithms on workloads in mechanics, contact, thermal-fluid systems...SeniorFull timeRemote work$10k
...Ramp is the place to do it.About the RoleWe’re looking for a Senior Applied Scientist to help drive the future of credit applied science at Ramp.... ..., deployment, monitoring, and iterationInvestigate and evaluate new data sources, including structured and unstructured data...SeniorFull timeWork at officeHome officeFlexible hours$145k - $175k
...an inclusive environment for all employees.About the role:As a Senior Computational Biologist at Immunai, you will be working in a fast... ...’s capabilities, including through the effective utilization of LLM-based toolsChampion Immunai core valuesRequirementsRequired qualifications...Senior$30 - $50 per hour
...an AI Researcher to support end-to-end research for modern AI systems. This remote role involves designing experiments, defining evaluation protocols, and improving evaluation rigor for large language models. Key responsibilities include developing AI research experiments...Remote jobHourly pay
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Senior LLM Evaluation Scientist. Be the first to apply!
Related searches
- scientist ii New York, NY
- scientist 1 New York, NY
- image scientist New York, NY
- downstream processing scientist New York, NY
- entry level research scientist New York, NY
- qc scientist New York, NY
- research scientist New York, NY
- analytical scientist New York, NY
- research scientist - biology New York, NY
- genomics scientist New York, NY