Physics Researcher for AI Model Evaluation

Create and verify human-quality reference solutions for the CritPt benchmark (arXiv:2509.26574v3), a frontier research-level physics benchmark. The role produces fully human-verified reference data used to evaluate large language model performance on frontier physics reasoning. Work includes solving CritPt research-level problems end-to-end, auditing other experts'' solutions, and adjudicating between parallel solution attempts to determine the golden reference.

Physics subdomains covered

• High Energy Physics and Mathematical Physics
• Biophysics and Statistical Physics
• Condensed Matter and AMO
• Gravitation, Cosmology, and Astrophysics
• Quantum Information
• Optical Properties of Materials
• Magnetic Materials
• Measurements in Quantum Mechanics

• Solve research-level physics challenges end-to-end, with verifiable derivations, runnable code, and peer-reviewed references.
• Decompose challenges into standalone checkpoint sub-problems that require genuine physical reasoning and can be independently verified.
• Author Python answer templates that include automated grading functions for symbolic and numerical answers.
• Audit submitted solutions for correctness, scope, and soundness of method, providing actionable feedback across iterations.
• Adjudicate between parallel solver attempts and decide which solution becomes the golden reference for a problem.
• Document detailed chain-of-thought reasoning, specify error tolerances, present equivalent symbolic forms, and supply verification test cases.

• Solver track: PhD or postdoc in the relevant subfield, senior PhD student minimum.
• Auditor track: Postdoc or junior professor in the relevant subfield, PhD minimum.
• Adjudicator track: Full professor or industry research principal investigator in the relevant subfield, senior postdoc or junior professor minimum.
• Hands-on familiarity with at least two canonical methods of the target subfield, demonstrated through publications, broader coverage preferred.
• Provide 3 to 5 representative publications, with arXiv ID or DOI, ideally within the last approximately 5 years and in the target subfield.
• Working proficiency with LaTeX, Python, Jupyter, and SymPy.
• Strong written English, B2, C1, or C2 level minimum; native or near-native preferred.

• Location: Remote.
• Employment type: hourly.
• Expected commitment: approximately 10 hours per week, sustained across an 8 to 10 week window per task pool.
• Work is asynchronous.

• Pay range: $80 to $135 per hour, based on role and demonstrated expertise.

• Candidates must hold the academic or research standing specified under Qualifications for the track they apply to.
• Applicants must be able to provide the requested publications (arXiv ID or DOI) and demonstrate working proficiency with the listed tools.
• Strong written English is required to prepare the human-verified reference solutions and feedback.