Staff Data Scientist, LLM Data Quality & Evaluation
Cohere
Cohere is looking for a Member of Technical Staff in Data Analysis and Evaluation to ensure the quality and performance of large language models. The role involves designing data collection tasks, collaborating with teams, and applying statistical methods for data evaluation. The ideal candidate will have strong software engineering and statistical skills, along with experience in machine learning frameworks. Applicants from diverse backgrounds are encouraged to apply! #J-18808-Ljbffr Cohere
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Staff Data Scientist, LLM Data Quality & Evaluation in San Francisco, CA vacancy
- ...enterprise AI company, is hiring a Member of Technical Staff in Data Analysis and Evaluation. You will design data-collection tasks, apply statistical... ...Collaborate with researchers and engineers to improve dataset quality and model performance, including distributed training of...Quality
$210k - $385k
A leading financial technology firm seeks a Data Scientist to architect evaluation pipelines and improve answer quality through innovative evaluations. Candidates should possess a PhD or MS in a technical field with at least 4 years of relevant experience, strong skills...Quality$238k - $302k
...across 15+ U.S. states. The Large Model Evaluation team is at the nexus of Waymo’s AI... ...model development and deployment. Build data pipelines for signal discovery, data labeling... ..., etc. Experience evaluating the quality of ML models Demonstrated experience taking...QualityFull timeRemote workWaymo
San Francisco, CA23 hours ago$164.7k - $339.08k
...Pinterest is seeking an experienced Staff Data Scientist to join our Finance &... ...catching and resolving data-quality issues.Partner broadly and... ...backtesting and forecast-accuracy evaluation, and seasonality analysis (y... ...applying modern AI/LLM tooling to real workflows —...QualityWork at officeLocal areaRelocationRelocation package$210k - $385k
...time Location Type Hybrid Department Data Science Compensation $210K - $385K... ...of users daily with reliable, high-quality answers grounded in an LLM-first search engine and our... ...Responsibilities Architect and maintain automated evaluation pipelines to assess answer quality...QualityFull timeLocal area$172.5k - $260.1k
...RoleWe are looking for an experienced Staff Data Scientist to support Slack's Go-to-Market (GTM)... ...methods to understand customer behavior, evaluate strategic initiatives, and optimize... ...learning.Experience developing production-quality analytical models and partnering with...QualityFull time$238k - $302k
...15+ U.S. states. Rigorous behavioral evaluation of the Waymo Driver is a critical part of... ...will lead key initiatives for measuring the quality and trustworthiness of the behavior... ...In this hybrid role you will report to a Data Science Lead. You will: Lead a roadmap...QualityFull timeRemote workLatent Logic
San Francisco, CA4 days ago$164.7k - $339.08k
...relevant experience in data science, analytics engineering... ...applying modern AI or LLM tooling to real... ...ready language. We want staff-level autonomy with the... ...and resolve data quality issues that affect recurring... ...hiring a Staff Data Scientist for our Finance & Business...QualityFull timeWork at officeRelocation packagePinterest, Inc.
San Francisco, CA1 day ago$160k - $200k
The Opportunity As a Staff Data Scientist on the Clinical Performance team, you will be the lead... ...providers. While your primary focus is evaluative, you will be a key player in the broader... ...forecasting engines that predict our quality performance across various value-based...Quality$212k - $265k
...better mental health outcomes. As a Staff Data Scientist, Product Analytics , you will be a senior... ...will influence what we build, how we evaluate it, and how we scale what works. This... ..., online experiments, and human-quality signals when needed. Build scalable analytical...QualityWork from homeFlexible hours- ...research for their next generation of LLM products. Join us if you: Wish to work... .... Responsibilities Own LLM evaluation processes and methods with a focus on... ...safety vulnerabilities. Generate high quality synthetic data, curate labels, and conduct rigorous benchmarking...QualityLocal areaShift work
$100 - $150 per hour
...Summers , and Jack Dorsey . Position: Data Scientist Talent Network Type: Contract... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work against established criteria to ensure quality and consistency. Provide detailed...QualityContract workSummer workRemote workMercor
San Francisco, CA1 day ago- ...Dynamo AI is seeking a candidate to lead LLM evaluation and benchmarking in San Francisco, California. You will generate high-quality data and develop innovative methods for assessing the safety and helpfulness of LLMs. The role requires domain knowledge in evaluation...Quality
Capitolis
San Francisco, CA2 days ago - .... About the Role As a Senior Software Engineer (AI Data & Evaluation) at Mercor, you will be at the core of building the data infrastructure... ...of frontier AI models. Our team's mission is to develop high-quality data types that push frontier models forward and drive the AI...QualityFull timeWork at officeRelocation package
Mercor
San Francisco, CA23 hours ago $195k - $257.5k
...everyone is a stakeholder.What you’ll be responsible forAs a Staff Data Scientist at Circle, you will partner closely with product, payments,... ...problem-solving skills with strong attention to detail and data quality.Preferred RequirementsDeep experience working with payments...QualityLocal areaFlexible hours$164.7k - $339.08k
...contributor, shaping how Pinterest's Ads organization leverages data for strategic decision-making and operational excellence. You'll... ...and identifying new opportunities.Ensure Data Trustworthiness and Quality: Establish and uphold high standards for data integrity,...QualityWork at officeLocal areaRelocationRelocation package$155k - $189k
...the Role:Gusto is looking for a highly skilled and motivated Staff Data Scientist to join our Time Products Data Science team. In this role,... ...highlight trade-offs and limitations based on sample size and data quality.Execution: Deliver multiple high-impact projects, balancing...QualityFull timeWork at officeLocal area2 days per week3 days per week$164.7k - $339.08k
...rhythms, executive decision-making, and strategic planning.Lead and mentor. Guide the work of at least two data scientists, raising the bar on technical quality, execution, and impact through candid, continuous feedback and coaching.What we’re looking for:8+ years of combined...QualityWork at officeLocal areaRelocationRelocation package$164.7k - $339.08k
...do:Develop measurement frameworks and metrics to drive data-driven decision making, optimizing for the full user journey... .... Leadership: Mentor the scope of work for other data scientists, demonstrating high-quality output of both yourself and others for whom you mentor....QualityWork at officeLocal areaRelocationRelocation package$238k - $302k
...Francisco, is seeking a skilled professional to lead methodologies for evaluating its autonomous driving technology. This hybrid role involves... ...) and have at least seven years of industry experience in data science. The position offers a competitive salary range of $238...Quality$164.7k - $339.08k
...production forecasting or time-series models on web-scale data. We need a proven history of delivering flexible,... .... You will lead and mentor at least two data scientists, raising the standard for technical quality, execution, and impact through ongoing feedback and...QualityFull timeRemote workRelocation packageFlexible hoursPinterest, Inc.
San Francisco, CA1 day ago$164.7k - $339.08k
...methods to solve real-world problems on web-scale data ~ Strong foundation in statistics, machine learning... ..., and insightful analysis Mentor other data scientists by helping define scope and raising the quality of output across the team Technologies: AI...QualityFull timeWork at officeRemote workRelocation packagePinterest, Inc.
San Francisco, CA1 day ago- B Capital is seeking a data engineer to ensure high data quality for training AI models. You will own the upstream data quality for LLM post-training and design automated QA methods in a collaborative environment. Ideal candidates will have strong engineering skills, a...Quality
- Reflection in San Francisco is looking for a data engineer dedicated to enhancing data quality for LLM pre-training. Your role will involve collaborating with world-class researchers to establish high standards for data collection and processing. You will design automated...Quality
$230k - $284k
...billions in simulation across 15+ U.S. states. Waymo’s Product Data Science team works cross-functionally with Engineering, Product... ...collaborates on high-impact projects across the company - from driving quality and operational efficiency to market analysis and rider...QualityFull timeWork experience placement- Carnaby Fox is seeking a Member of Technical Staff (AI Research) in San Francisco to help shape the research direction... ...collaborate with world-class researchers to design experiments, evaluate LLMs, and improve data quality for high-stakes AI benchmarks. The role emphasizes...Quality
$218.4k - $273k
...Scale has been the leading AI data foundry, helping fuel the most... ...building upon our prior model evaluation work with enterprise customers... ...requires not only expertise in LLM agents and planning algorithms... ...Our products provide the high-quality data and full-stack...QualityFull time$202k - $282k
The Data Science team at Asana is pivotal in fulfilling our mission by fostering a data... ..., setting the technical bar for modeling quality, code rigor, data pipeline architecture,... ...mentorship and guidance to a team of data scientists at varying levels, helping them navigate...QualityWork at officeWork from home- ...stack: improving the reliability, quality, and user experience of... ...product development with core data science, ML, and analytics work... ...product experiences, and the evaluation and observability systems that... ..., human review workflows, and LLM-as-judge evaluation patterns.Analyze...QualityWork at office3 days per week
- ...applying machine learning and data mining techniques to... ...-store technologies, LLM evaluationExceptional... ...WHAT YOU'LL DOAs a Data Scientist II, you will... ...analysisDesign, implement, and evaluate models—from traditional... ...and assess LLM output quality / hallucinationsDeploy...QualityApprenticeshipWork at officeLocal areaEasy work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Staff Data Scientist, LLM Data Quality & Evaluation. Be the first to apply!
Related searches
- assistant scientist San Francisco, CA
- data scientist machine learning engineer San Francisco, CA
- principal data scientist San Francisco, CA
- part time data scientist San Francisco, CA
- data scientist San Francisco, CA
- entry level data scientist San Francisco, CA
- senior data scientist San Francisco, CA
- ai data scientist San Francisco, CA
- python data scientist (contract) San Francisco, CA
- entry level data scientist remote San Francisco, CA