Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Evaluation Scientist Real-World ML Research

Arena Intelligence, Inc.

Arena Intelligence, Inc. in San Francisco seeks machine learning scientists to design and analyze experiments that reveal what makes AI models useful, trustworthy and capable through human preference signals. You will work with engineers, product teams, and researchers to develop methods for comparing models and disentangling performance factors.

This role is deeply interdisciplinary. You’ll contribute to open-ended questions, rigorous evaluation, and research with real-world impact while

#J-18808-Ljbffr
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the AI Evaluation Scientist Real-World ML Research in San Francisco, CA vacancy
  •  ...Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. The Data Flywheel...  ...with researchers, engineers, customers, partners, vendors, and the open-source community to drive real-world impact. #J-18808-Ljbffr... 
    Suggested

    Sierra Ventures

    San Francisco, CA
    1 day ago
  • $160k - $220k

     ...the RoleWe’re looking for an AI Research Scientist to advance the...  ...architectures, new training or evaluation techniques, long-horizon research...  ...and engineers on rigorous ML research practices.External...  ...evaluation, data governance, and real-world deployment constraints.What... 
    Suggested
    Temporary work
    Work at office
    Monday to Friday
    Monday to Thursday

    Sprinter Health

    San Francisco, CA
    1 day ago
  • $199k - $325k

     ...WRITERWRITER is where the world's leading...  ...orchestrate AI-powered work....  ...About the roleAI research at WRITER isn't...  ...an AI research scientist, you'll be at...  ...model training, evaluation, and production...  ...on complex, real-world enterprise...  ...years of hands-on ML research... 
    Suggested
    Full time
    Work at office
    Local area

    Writer

    San Francisco, CA
    4 days ago
  • $184k - $274.6k

     ...Meet the TeamAt Foundation AI, we are leading frontier AI research across Cisco. Our mission...  ...techniques that solve real-world problems at scale.We explore...  ...inference optimization, evaluation techniques, and data...  ...practices, and common AI/ML librariesExperience with... 
    Suggested
    Full time
    Temporary work
    Local area
    Flexible hours

    CISCO Systems

    San Francisco, CA
    1 day ago
  •  ...Principal AI Research Cybersecurity Scientist (Bay Area) We are seeking a...  ..., benchmarking, and evaluation. The ideal candidate...  ...at leading AI/ML conferences such as...  ...with machines. In a world where AI has fundamentally...  ...with low-latency Real-Time Guardrails and... 
    Suggested

    Alice

    San Francisco, CA
    10 hours ago
  •  ...bringing frontier AI into clinical practice...  ....S. X-rays. Our research team brings...  ...continuously enriched by real-world clinical data. Together...  ...or Research Scientist to advance how we curate data and evaluate frontier AI models...  ...evaluation systems, or ML research infrastructure... 

    Alt

    San Francisco, CA
    2 days ago
  • $143k - $219k

    The BasicsTanium is seeking an AI Research Scientist. This role reports to our Sr. Director AI...  ...product teams to design, develop, and evaluate AI and ML models that power intelligent...  ...intersection of cutting-edge AI research and real-world cybersecurity applications at... 
    Full time
    Live in
    Work at office
    Worldwide
    Flexible hours
    3 days per week

    Tanium

    Emeryville, CA
    4 days ago
  •  ...Omnifold is seeking a talented Data Scientist or Applied AI Scientist to work directly on real customer problems, understanding data and operational challenges...  ..., and become a trusted technical partner during evaluations, pilots, and implementations, helping customers... 

    Omnifold

    San Francisco, CA
    4 days ago
  •  ...infrastructure frontier AI labs build internally,...  ...compute, environments, evaluations, secure sandboxes, high...  ...tasks like autonomous research, and the full-stack platform...  ...of frontier research, real infrastructure, and go-...  ...evaluation systems, or ML workflows used by... 
    Remote work
    Visa sponsorship
    Relocation package
    Flexible hours

    Wwshemi

    San Francisco, CA
    4 days ago
  • $150k - $350k

     ...Collate is an AI document generation...  ...of Lever. Our AI researchers, engineers, and designers...  ...of a company with world‑changing potential...  ...an AI Research Scientist to push the...  ...in the lab become real solutions in the world...  ...production. Develop evaluation frameworks to... 

    Collate

    San Francisco, CA
    1 day ago
  •  ...MakerMaker in San Francisco seeks researchers to design, build, and evaluate autonomous multi-agent systems that perform open-ended ML research and discovery. You’ll shape how agents...  ...frameworks, and shipping into production with real ownership and responsibility for... 

    MakerMaker

    San Francisco, CA
    1 day ago
  • $270k - $340k

    Principal AI Research Scientist, Research Director - AI ScalingP-1227About Databricks...  ...data teams to solve the world’s toughest problems, from...  ...approaches.Optimize end‑to‑end ML systems for distributed...  ...state‑of‑the‑art methods and evaluating trade‑offs in quality, latency... 
    Local area
    Worldwide

    DataBricks

    San Francisco, CA
    4 days ago
  • $246.5k - $339k

     ...discover the best products from around the world to sell in their stores. With the right...  .... Our highly skilled team of Applied AI/ML Scientists specialize in developing algorithmic solutions...  ...using machine learning to solve real-world problems.Experience with relevant... 
    Work experience placement
    Work at office
    Local area
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    5 days ago
  • $200k - $300k

     ...exponential progress in AI converges with...  ...investments, and in-house research programs. We accept...  ...lack feedback from real biology....  .../Staff AI Research Scientist to join our foundational...  ...loops, and evaluation metrics. Bridge...  ...combination of frontier ML work directly coupled... 
    Immediate start
    Remote work

    Astera Institute

    San Francisco, CA
    10 hours ago
  • $211k - $290.5k

     ...best products from around the world to sell in their stores....  ...highly skilled team of Applied AI/ML Scientists specialize in developing...  ...product they see on Faire are the real thing. That means matching...  ...building and validating LLM evaluation pipelines, including prompt... 
    Work experience placement
    Work at office
    Local area
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    1 day ago
  • $211k - $290.5k

     ...products from around the world to sell in their...  ...roleAs a Senior Applied AI/ML Scientist on the Search team, you...  ...site). You will advance real-time search and...  ...browsing, filtering, and evaluation.Share best practices regarding...  ...—moving quickly from research papers to prototypes... 
    Work experience placement
    Work at office
    Local area
    Immediate start
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    2 days ago
  • $211k - $290.5k

     ...products from around the world to sell in their stores...  ...— Faire’s user facing AI bet within the...  ...As a Senior Applied AI/ML Scientist on the Compass team, you...  ...quality through data, evaluation, and modeling, while shipping...  ...helping you grow in yours.Real rewards. Competitive... 
    Work experience placement
    Work at office
    Local area
    Immediate start
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    1 day ago
  • $210k - $350k

     ...Scientific builds and deploys AI scientist agents to accelerate...  ...Member of Technical Staff, AI Research Scientist at Edison Scientific...  ...and scientists around the world. This role is on-site at...  ...of strong track record of ML research on real‑world problems in (but not... 
    Full time
    Work at office

    Edison

    San Francisco, CA
    10 hours ago
  • $149k - $350k

     ...iterating with AI. From idea to product...  ...together in real time from anywhere in the world. If you’re excited...  ...for applied scientists with a Machine Learning...  ...and applied research in this area....  ...design and build AI/ML models and...  ...systems Build evaluation systems to measure... 
    Full time
    Temporary work
    Remote work
    Work from home

    The available sources do not contain information about the c...

    San Francisco, CA
    10 hours ago
  • $211k - $290.5k

     ...Senior Applied AI/ML Scientist - Compass Faire is a technology wholesale...  ...products from around the world to sell in their stores. With...  ...agent quality through data, evaluation, and modeling, while shipping...  ...helping you grow in yours. Real rewards. Competitive pay, equity... 
    Work experience placement
    Work at office
    Local area
    Immediate start
    Remote work
    Monday to Friday
    Flexible hours
    3 days per week

    Faire

    San Francisco, CA
    2 days ago
  • $405k

     ...Engineer, Staff: Applied AI, Science &...  ...group of committed researchers, engineers, policy...  ...breakthroughs and real-world applications. Claude...  ...Claude to the scientists and engineers...  ...harnesses, integrations, evaluations, and working...  ...Experience with ML research, RL environments... 
    Work at office
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    3 days ago
  • $263.6k - $395.4k

     .... We want to redefine the world’s relationship with money...  ...happy. The Role We build AI products, and the quality of our evaluations sets the ceiling for how...  ...predicting what happens with real users. Our evaluation...  ...product, engineering, data, and ML teams to make evaluation... 
    Work at office
    Local area
    Remote work
    Flexible hours

    Block Inc

    San Francisco, CA
    3 days ago
  • $164.7k - $339.08k

     ...people around the world come to our platform...  ....At Pinterest, AI isn't just a feature...  ...here.As a Data Scientist in our Core organization...  ..., Design, Research, Product Analytics...  ...scientific rigor into the real world products...  ...robust, scalable ML and evaluation frameworks—spanning... 
    Work at office
    Local area
    Relocation
    Relocation package

    Pinterest

    San Francisco, CA
    3 days ago
  • $300k

     ...Team: Grindr is an AI-native platform powering...  ...see. As a Staff Data Scientist at Grindr, you will be...  ...scientists, data engineers, and ML/AI engineers into a...  ...Design and evaluate experiments to measure...  ...that function well in the real world would be a nice to have... 
    Work at office
    Immediate start
    Worldwide
    Flexible hours

    Grindr

    San Francisco, CA
    2 days ago
  • $300k

     ...Team: Grindr is an AI-native platform powering...  ...As a Senior Staff Data Scientist at Grindr, you will lead...  ..., data engineers, and ML/AI engineers into a single...  ...for designing and evaluating experiments to measure...  ...that function well in the real world would be a nice to have... 
    Work at office
    Immediate start
    Worldwide
    Flexible hours

    Grindr

    San Francisco, CA
    2 days ago
  • $182k - $228k

     ...looking for a Staff Data Scientist who combines deep...  ...full lifecycle. From research, development and deployment...  ...audiences. Shape and evaluate product initiatives...  ...for reproducibility, ML ops, and scientific rigor...  ...your models influence real-world outcomes. 7+ years of... 
    Full time
    Work at office
    Local area
    3 days per week

    Turo Inc.

    San Francisco, CA
    10 hours ago
  •  ...join our global ML organization. In...  ...engineers around the world. You will play a...  ...future of ML/AI in the video...  ...the before/after evaluation numbers, and why...  ...retraining — with real latency budgets, not just research notebooks....  ...For the Data Scientist position, we will... 
    Flexible hours

    Logoslabs

    San Francisco, CA
    1 day ago
  •  ...Origin is building physical AI for the built world. Our robots autonomously...  ...Jetson AGX Orin. Every research project will have a...  ...detection. Design offline evaluation metrics that predict real-world finishing quality...  ...MS/PhD in CS, Robotics, ML, or equivalent... 

    Origin

    San Francisco, CA
    1 day ago
  •  ...ML/AI Research Engineer — Agentic AI Lab (Founding Team) Location: San...  ...Backed by 8VC, we're building a world-class team to tackle one of...  ...lead the design, training, evaluation, and optimization of agent-native...  ...pipelines integrated with real-time or contextual data Familiar... 
    Full time

    Fabrion

    San Francisco, CA
    10 hours ago
  •  ...Lila Sciences is assembling a Safety AI team in Cambridge, MA, London, and San Francisco...  ...to develop tests and tools that evaluate safety-relevant capabilities of scientific...  ...scientific AI systems. The role spans Research Scientist and Research Engineer profiles,... 

    Lila Sciences, Inc.

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Evaluation Scientist Real-World ML Research. Be the first to apply!