LLM Benchmark Lead Research Scientist
Vals AI
Vals AI in San Francisco is seeking exceptional researchers and research engineers to design and build the next generation of AI benchmarks. You will create high-impact, challenging evaluations that push the boundaries of what we can measure in foundation models. This role is perfect for someone with deep research expertise who wants to see their work directly influence how the world evaluates AI systems. We are building the standard for evaluating the ability of LLMs to perform real-world #J-18808-Ljbffr Vals AI
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the LLM Benchmark Lead Research Scientist in San Francisco, CA vacancy
- Camb in San Francisco is searching for a Lead LLM R&D Researcher. This full-time position involves leading innovations in natural language processing and colloquial translation as well as collaborating with R&D teams. The ideal candidate will have a PhD or equivalent experience...SuggestedFull time
- ...also happen to be the world's leading generative AI studio—we're... ...We’ve developed an in-house LLM storytelling system that blends... ...Project Astra, and top-tier AI researchers. As an early member of this... ...saving techniques, scalable benchmarking and checkpoint selection, are...SuggestedWork at officeVisa sponsorship
- Bay Area Full time Lead LLM R&D, innovate in zero-shot performance and colloquial translation, collaborate, and publish research in top conferences. Join Camb.ai's pioneering AI team... ...just innovating; we're setting SOTA benchmarks. Backed by elite global VCs, partnering...SuggestedFull time
- Modal is seeking a research-focused post-training LLM scientist to join our team in advancing end-to-end post-training workflows, including training, deployment... ...in production. You will work alongside the research lead to identify high-impact bets and shepherd them from...Suggested
- Handshake is hiring a Research Scientist, Human-AI Interaction to define how AI systems support real human work at scale. You will lead research at the intersection of HCI, LLMs, and benchmarking, shaping evaluation for enterprise AI copilots and task frameworks used globally...Suggested
- Fleet AI, Inc. is seeking a Research Scientist to join their core research team in San Francisco. This role focuses... ...own research projects, and collaborating with leading labs. Key responsibilities include generating benchmarks to evaluate frontier models, automating...
- Morph builds models that improve coding agents, including the fastest LLM code editing engine on the market—speculative decoding and more—delivering 10,500 tok/sec code edits. Our stack powers high‑throughput AI workflows for coding apps, devtools, PR bots, and IDEs. You...
- ...Annotate is searching for a Strategic Projects Lead/AI Operations Manager who will take ownership of... ...managing the end-to-end delivery of large LLM and Gen AI data programs, engaging directly with clients and researchers. The ideal candidate will have strong technical...
- A biotech company focusing on AI in drug discovery seeks a highly skilled AI Research Scientist to drive the development of next-generation AI models used in drug discovery workflows. The ideal candidate possesses a Master's or PhD in a relevant field along with at least...
- Artificial Analysis, Inc. in San Francisco is seeking a Member of Technical Staff to lead its new robotics vertical. This role entails defining benchmarking methodologies and producing leaderboards to evaluate robotics capabilities. Ideal candidates will possess a strong...
$357k
...remote workers Responsibilities Workato's AI Research Lab is seeking an exceptional Lead AI Research Scientist to join our growing team and define the next generation... ...-on experience with PyTorch or JAX and modern LLM frameworks. ~ Proven track record of...Work at officeRemote workFlexible hours$181.1k - $318.4k
Apple Inc. is seeking a Senior Applied Researcher in San Francisco to drive innovation in Generative AI and advanced NLP systems. This role involves architecting LLM-powered systems and leading research in large-scale representation learning, semantic modeling, and more...$300k - $320k
...growing group of committed researchers, engineers, policy... ...exceptional Research Scientist to join our Life Sciences... ...training objectives, benchmarks, and agentic workflows... ...Experience with LLM post‑training: RLHF, RL... ...target identification, lead optimization, ADMET modeling...Visa sponsorship$300k
...develop new reinforcement learning algorithms for post-training language models in San Francisco. The successful candidate will own a research agenda, establish baselines for evaluating efficiency, and collaborate with peers to innovate algorithmic solutions. Candidates...- ...brilliance. Founded by engineers and researchers who dreamed of building the... ...evaluations that go beyond benchmarks. We've run a profitable... ...funding. The Role As a Research Scientist focused on data, you’ll work at the cutting edge of LLM development by designing the datasets...
$250k
...better. We work with the world's leading labs to design high signal... ...that go beyond static benchmarks. We are a small, early team (... ...measured. Working directly with research teams at top AI labs, you’ll... ...instincts and familiarity with LLM training pipelines, RLHF/RLVR...- ...and genuine care. The Role Research at Aristotle works on the open... ...reward signal, and existing benchmarks test single turns in toy scenarios... ...with one of the co-founders leading research, and with data that... ...‑on experience with some of: LLM evaluation, post‑training/...Full timeContract workSummer workFlexible hours
$350k - $420k
Research Scientist, Human AI Interaction Handshake is the career network for... .... This unique value is leading to unparalleled growth; in 2... ...Language Models, and task‑level benchmarking. You will operate at the frontier... ...production, including: LLM‑powered copilots and...Full timeFreelanceInternshipWork at officeRemote workWorldwideFlexible hours$225k - $400k
ABOUT THE ROLE This is a research-driven, high-impact role for ML researchers who want to... ...into working prototypes. Evaluation & Benchmarking - Design novel evaluation frameworks, datasets... ...worked on advanced ML problems (e.g., LLM pre‑training and post training,...H1bRelocation$250k
...partner with the world's leading labs to design high‑... ...evaluations that go beyond static benchmarks. We're a small, early... ...out our post‑training research team and hiring 2-3 Research Scientists to work together on this... ...Strong familiarity with LLM training and evaluation...InternshipShift work- ...is for an experienced scientist who thrives both in innovating... .... What You’ll Do Lead design and... ...deep content extraction. Research, evaluate, and integrate... ...product impact. Develop new benchmarks, datasets, and evaluation... ...use cases, including LLM orchestration, context...
- ...agents, and we are hiring a Research Scientist to advance the neuro-symbolic... ...scale. What You'll Do Lead research initiatives in neuro... ...with full observability and benchmarking. Engage with the Bay Area academic... ..., etc.). Familiarity with LLM reasoning, retrieval-augmented...Work at officeRelocation
- A leading transcription service provider in San Francisco is looking for a highly skilled Computer Scientist to enhance the performance of Large Language Models (LLMs), focusing on Retrieval... ...engineering and knowledge of various LLM frameworks, making it ideal for...
- P-1 AI in San Mateo, CA is looking for an exceptional AI Research Scientist/Manager to join their elite team. This role merges cutting-edge AI research with hands-on engineering, perfect for someone who excels at the convergence of ideas and implementation. The ideal candidate...
- About the role AI research at WRITER isn't just about publishing... .... As an AI research scientist, you'll be at the... ...applicants What you’ll do Lead an independent, high‑... ...Build novel evaluation benchmarks and methodologies that... ...— including LLM‑as‑judge frameworks, synthetic...Full timeLocal areaFlexible hours
- WRITER, located in California, is searching for a Senior AI Research Scientist to lead innovative projects centered on large language models and agentic reasoning. The role emphasizes the exploration of advanced AI techniques and their application in enterprise AI solutions...
$204k - $259k
...simulation across 15+ U.S. states. The mission of the Waymo Applied Research team is to develop machine learning solutions addressing open... ...Alphabet. In this hybrid role you will report to a Technical Lead Manager. You will Conduct applied foundation model research and...Full timeRemote work- Senior Research Scientist, Model Evaluation Cohere | Cohere | Posted Mar 2 Full-time Negotiable... ...methods and infrastructure to measure LLM progress. As a Senior Research Scientist... ...Will Create ambitious new evaluation benchmarks that push the limits of what our models...Full timeWork at officeRemote workFlexible hours
$216k - $270k
SCALE LABS, RESEARCH SCIENTIST — FRONTIER RISK EVALUATIONS As the leading data and evaluation partner for frontier AI companies, Scale... ...in crafting evaluations and benchmarks, or a background in data science roles related to LLM technologies. * Experience with red...Full time$100 per hour
...AI experience is required. In this role, you'll design physics problems, evaluate AI performance, and collaborate with researchers to improve benchmarks. The ideal candidate will have strong reasoning and problem-solving abilities, along with clear communication skills....Remote jobContract workFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to LLM Benchmark Lead Research Scientist. Be the first to apply!
Related searches
- support scientist San Francisco, CA
- analytical scientist San Francisco, CA
- safety scientist San Francisco, CA
- hplc scientist San Francisco, CA
- drug safety scientist San Francisco, CA
- research scientist - biology San Francisco, CA
- senior principal scientist San Francisco, CA
- water quality scientist San Francisco, CA
- qc scientist San Francisco, CA
- decision scientist San Francisco, CA