AI Scientist Benchmark Fellow

What’s the next AI frontier?

Scientific discovery, automated. The question is, “how do you judge if an AI model is doing ‘good research’”. The answer comes from researchers who use AI models on real research. So, we're seeking one Fellow per STEM field who wants to build their field's first benchmark and put their name on it.

Core Responsibilities 

• Validate results of the work produced by AI models’ paper reproduction and simulations.
• Map out your field, including the most important research areas and the taxonomy within.
• Set the standards for refining AI traces into benchmark samples; create benchmarks to evaluate frontier AI models against.

Authorship

• Your benchmark will be published as a citable public release, including a paper and an open eval set.  You’ll be named a lead author. 
• Bonus: Get more citations for your papers.

Requirements

• PhD students or postdocs in biology, medicine, neuroscience, materials science, chemistry, physics, mechanical engineering, or chemical engineering.
• Have published a paper in your field; can judge a paper in your field and catch author’s errors.
• Currently doing AI for Science research in their fields. (For example, you are using AI agents like Claude Code or Codex for 20+ hours per week in your research).

How it works

• Can be remote or in-person.
• You can set your hours. You can take a break if there’s an academic deadline.
• $50-100/hr (up to $250K annualized). If you’re in the US, F1/visa-friendly structures are available.
• You get to work directly with our founders and with researchers at frontier AI labs.

About OpenRefinery

Our mission is to advance scientific research via AI advancement. We scaled Turing from $0 to 2.2B (Series E). The cofounders Kai, Alex, and Char led ML programs Facebook AI Research (FAIR) and Zoox, exited two companies before, and built a $300M ARR data business together.

Backed by Zetta Ventures (invested in Kaggle and Domino Data Lab), Chair of the MIT Board of Trustees, Audacious Ventures (investors in Reflection AI and Decagon), and AI researchers from OpenAI, Anthropic, Google DeepMind, Thinking Machines, Nvidia, and the United Arab Emirates’ AI institute. 

We partner with researchers at Stanford, MIT, Caltech, ETH Zürich, UPenn, Berkeley, NUS, and more.

How to Apply

Email View email address on nature.com with your field, how you use AI in your research, and one thing models consistently fails in your field. Résumé/CV is optional; you can just share your academic profile.