Research Scientist — AI Safety Benchmarks & Evaluation
Plato
Plato, an applied AI research lab in the San Francisco Bay Area, is hiring an individual contributor researcher to design safety environments and benchmarks. You will establish a new research direction, measure AI R&D capabilities, and work with engineers to turn designs into runnable environments with clear scoring and limits. You will investigate reward hacking, deception, and other misaligned behavior, owning the research agenda while collaborating with lab partners. Relocation encouraged. #J-18808-Ljbffr Plato
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Research Scientist — AI Safety Benchmarks & Evaluation in San Francisco, CA vacancy
- ...Lila Sciences is assembling a Safety AI team in Cambridge, MA, London, and San Francisco to develop tests and tools that evaluate safety-relevant capabilities of scientific... ...scientific AI systems. The role spans Research Scientist and Research Engineer profiles, focusing...Suggested
Lila Sciences, Inc.
San Francisco, CA14 hours ago $140k - $200k
...The RoleAs a Research Engineer (RE) or Research Scientist (RS) at CAIS, you'll lead... ...research that advances the safety and reliability of frontier AI systems. This is the... ...to train and evaluate models at scale, and... ...maintain datasets and benchmarks.Run distributed training...SuggestedWork at officeLocal areaCenter for Ai Safety
San Francisco, CA3 days ago$150k - $250k
About Distyl AI Distyl is an applied AI technology company partnering... ...social organizations.We research and deploy technologies that... ...incremental improvements on benchmarks or optimize an existing... ...measured. Researchers design evaluation frameworks that capture reasoning...SuggestedWork at office3 days per weekDistyl AI
San Francisco, CA8 days ago$180k - $298k
...Research Scientist / Research Engineer, Automated Safety Cambridge, MA USA; London, UK; San Francisco... ...-dense, high-agency AI safety team at Lila that... ...engagement, safety-focused evaluations, safety systems to... ...investigation beyond fixed benchmarks. Build reproducible...SuggestedFull timeWork at officeLocal areaFlexible hoursLila Sciences, Inc.
San Francisco, CA1 day ago- ...Plato Plato is an applied research lab building the... ...infrastructure to train specialized AI agents. We turn real-... ...with the frontier: Build safety environments and benchmarks that research teams at leading... ...: Help set Plato’s safety evaluation agenda, from the research...SuggestedRelocation
Plato.ai
San Francisco, CA2 days ago - ...diagnostics by bringing frontier AI into clinical practice.... ...U.S. X-rays. Our research team brings experience... ...Engineer or Research Scientist to advance how we curate data and evaluate frontier AI models for medical... ...to developing rigorous benchmarks for radiology report...
Alt
San Francisco, CA1 day ago - ...Acceler8 Talent is hiring an Evaluation Researcher for a frontier AI startup in San Francisco, onsite. You will help define, measure and improve what... ...for real-time, multimodal AI systems, designing benchmarks and testing methodologies that assess speech, reasoning...
Acceler8 Talent
San Francisco, CA5 days ago $250k
...accelerate cancer research, improve construction site safety, digitize floor plans... .... We build our own benchmarks grounded in real... ...that is exploding as AI traverses the physical... ...As a Research Scientist at Roboflow Labs, you... ...findings into dataset and evaluation specifications, and...Work at officeRoboflow
San Francisco, CA4 days ago- ...Apart Research in San Francisco is seeking a Research Scientist for Manipulation Evals to design, build, and publish novel evaluations of AI harmful manipulation that inform EU AI Act enforcement. You will own the full pipeline and steer research directions with significant...Remote work
Apart Research
San Francisco, CA4 days ago - ...Mercor is hiring PhD and Master's scientists to author AI evaluation tasks for a new benchmark in scientific computing. You will author original, executable research problems that today's frontier models cannot solve. Domains require depth in biology with focus on ecology...Part timeImmediate start
Mercor Inc
San Francisco, CA2 days ago - ...accomplished Senior Responsible AI Researcher to lead state-of-the-art... ...frontier models, Trust & Safety (T&S), model evaluation, and adversarial... ...safety), and build safety benchmarks across Generative AI modalities... ...junior researchers, data scientists, and engineers. Bridge...
- Scale AI, Inc. seeks Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation. The role focuses on building benchmarks and diagnosing model failure modes in text and multimodal modalities within the GenAI...
$350k
...Research Engineer / Scientist, AlignmentSan Francisco, CAAbout AnthropicAnthropic's... ...interpretable, and steerable AI systems. We want AI to be... ...experimental research on AI safety, with a focus on risks from... ...coordination with third-party evaluators.Safeguards Research:...Work at officeVisa sponsorshipFlexible hoursAnthropic
San Francisco, CA5 days ago- ...OpenAI, Inc. is seeking researchers to advance safety training for its models. You will develop methods that improve model behavior after pretraining... ...to identify risks, design interventions, train models, evaluate outcomes, and launch safety improvements in OpenAI...
OpenAI, Inc.
San Francisco, CA4 days ago $216k - $270k
Scale Labs, Research Scientist — Safety Post TrainingAs the leading data and evaluation partner for frontier AI companies, Scale plays an integral role in understanding the capabilities... ...safety standards, evaluation benchmarks, and best practices.Ideally you’d have:...Full timeScale AI
San Francisco, CA4 days ago$165.6k - $207k
...the industry's leading AI labs to provide high quality... ...progress in GenAI research. We are looking for Research Scientists and Research Engineers with... ...RLHF, reward modeling) and evaluation. This role is on the... ...will focus on building benchmarks and diagnosing model failure...Full timeScale AI
San Francisco, CA4 days ago$200k - $375k
...About Snorkel Snorkel AI is the frontier AI... ...combine technology with research-driven AI data... ...development to create datasets, benchmarks, evals, and custom... ...looking for a Research Scientist to advance how high-quality... ...developing and evaluating new methods. Experience...Local areaRemote workSnorkel AI
San Francisco, CA5 days ago$150k - $200k
...do Contra Labs is a human-centered AI lab focused on creative and multimodal... ...creative preference datasets that power benchmarking, evaluation, and post-training for the world's... ...exists Contra Labs is expanding its research work with frontier AI labs across evaluation...Contra
San Francisco, CA5 days ago- ...Save “ Research Scientist - Frontier Benchmarks ” at Snorkel AI , track your applications, and ask the Job Agent to rank similar roles for you. At Snorkel... ...art datasets that drive frontier model training and evaluation based on current model performance and academic...Local area
JRNLClub
San Francisco, CA4 days ago - ...Roboflow is seeking a Research Scientist for Roboflow Labs in San Francisco to work at the intersection of benchmarks, models, and frontier labs, running failure analyses and turning those findings into data and evaluation specs. You will publish, extend RF100-VL,...
Roboflow, Inc.
San Francisco, CA3 days ago $197k - $262.5k
...or Remote, US Descript’s Research team builds the models behind... ...editor does. Underlord, our AI editing agent, reasons about... ...settled way to represent or evaluate editorial craft, whether a cut... .... Evaluation: design the benchmarks and evals that make editorial...Work at officeRemote workFlexible hoursMosaec
San Francisco, CA5 days ago$130k - $200k
...cost of living. Apart Research is hiring a Research Scientist for Manipulation Evals... ...build, and publish novel evaluations of AI harmful manipulation that... ...globally distributed AI safety research lab. We have... ...public deliverables (papers, benchmark releases, technical...Work at officeLocal areaRemote workHome officeFlexible hoursApart Research
San Francisco, CA4 days ago$199k - $325k
...enterprises orchestrate AI-powered work. Our... ...AI. About the roleAI research at WRITER isn't just about... ...world. As an AI research scientist, you'll be at the... ...through model training, evaluation, and production deploymentDesign... ...novel evaluation benchmarks and methodologies that...Full timeWork at officeLocal areaWriter
San Francisco, CA8 days ago$160k - $220k
...runway.About the RoleWe’re looking for an AI Research Scientist to advance the methodological... ...novel architectures, new training or evaluation techniques, long-horizon research bets... ...validation standards are higher than benchmark culture alone, and you are energized...Temporary workWork at officeMonday to FridayMonday to ThursdaySprinter Health
San Francisco, CA5 days ago- OpenAI is seeking researchers and engineers to advance frontier-model research with a focus on safety and alignment. You will train and evaluate models, identify misalignments, and develop scalable measurement and evaluation pipelines to turn failures into safety signals...Relocation package
- OpenAI is seeking a safety- and security-minded researcher or engineer in San Francisco to design and test practical... ...for increasingly capable AI agents. The role focuses on oversight... ...productionize AI controls, conduct red-team evaluations, and iterate mitigations based on...
$148k - $208k
...USA We're building a talent-dense, high-agency AI safety team at Lila that will engage all core teams... ...development, broader ecosystem engagement, safety-focused evaluations, safety systems to mitigate risks, and a research agenda that explores longer-term needs such as...Full timeWork at officeLocal areaFlexible hoursLila Sciences, Inc.
San Francisco, CA13 hours ago- ...Mercor is assembling a panel of chemistry and chemical safety experts to red-team frontier AI models. You will write challenging prompts at three levels (benign, dual-use, adversarial) and evaluate model responses against a defined policy standard, providing a reference...
Mercor Inc
San Francisco, CA5 days ago $268k - $384k
...building a talent-dense, high-agency AI safety team at Lila that will engage... ...engagement, safety-focused evaluations, safety systems to mitigate risks, and a research agenda that explores longer-term... .... We are seeking a materials scientist to map risk across the material...Full timeWork at officeLocal areaImmediate startFlexible hours- Key Talent Solutions seeks a Research Scientist - Post Training to advance how training data shapes... .... This hands-on role blends research, evaluation and collaboration with external... ...and RL experiments, build reproducible benchmarks, publish results, and translate findings...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Research Scientist — AI Safety Benchmarks & Evaluation. Be the first to apply!
Related searches
- research scientist machine learning deep learning San Francisco, CA
- image scientist San Francisco, CA
- health scientist San Francisco, CA
- regulatory scientist San Francisco, CA
- scientist biology San Francisco, CA
- drug safety scientist San Francisco, CA
- research associate scientist San Francisco, CA
- lead scientist San Francisco, CA
- principal applied scientist San Francisco, CA
- entry level research scientist San Francisco, CA