Particle & Nuclear Physics Expert for AI Benchmark Design

In this role, you will contribute to a groundbreaking project aimed at evaluating the capabilities of advanced AI systems in tackling complex scientific and engineering challenges. As a task designer, you will create sophisticated computational problems that assess whether AI can effectively utilize real scientific software for research-level tasks, including running simulations, interpreting results, designing experiments, and extracting valuable insights from data.

This position goes beyond typical data-labeling tasks. You will develop original, graduate-level problems rooted in authentic scientific workflows, rigorously test them against state-of-the-art AI models, and refine them to achieve the optimal level of difficulty.

• Design problems that require adept use of specialized scientific software, including tasks that test the AI''s ability to compute precise answers from well-defined setups and execute complex, multi-step workflows.
• Create more challenging problems where the AI must strategically plan a series of queries or experiments to uncover hidden information, necessitating thoughtful measurement, interpretation of partial results, and efficient narrowing of possibilities.
• Engage in a testing loop with cutting-edge AI models, continuously refining problems until they meet the desired difficulty level.

We are particularly interested in candidates with extensive, hands-on experience in:

• Particle & Nuclear Physics , proficiency with scikit-hep and related HEP Python tools for particle physics data analysis, cross-section computations, renormalization group calculations, and perturbative QCD. Experience with Monte Carlo event generation or collider phenomenology is advantageous.

The ideal candidate will possess graduate-level expertise (MS or PhD preferred) in the specified domain, complemented by practical experience using these tools. You should have a track record of writing code with these libraries to solve real research problems, along with a deep understanding of their limitations, edge cases, and the nuances that differentiate genuinely challenging problems from merely complicated ones.

In addition to domain expertise, strong candidates will exhibit puzzle design thinking, crafting problems where the challenge stems from intelligent reasoning rather than sheer computation, where multiple plausible approaches exist but only careful analysis leads to the correct solution, and where superficial pattern matching is insufficient.

• Graduate-level training in a relevant STEM field (MS, PhD, or equivalent research experience).
• Proven proficiency with at least one of the specified scientific software libraries, demonstrated through research publications, open-source contributions, or professional experience.
• Strong Python programming skills for writing problem setups, oracle functions, and solution validators.
• Ability to work independently and iteratively refine problem designs based on feedback.
• Comfortable operating in a Linux/terminal environment with remote compute sandboxes.
• Availability for at least 15, 20 hours per week.

• Experience across multiple relevant domains or tools.
• Familiarity with benchmark or evaluation design.
• Background in scientific teaching or exam/problem-set design.
• Experience with computational reproducibility and containerized environments.

Please note that this application process includes a coding assessment as part of the evaluation.