Computational Bayesian Statistics and Applied Mathematics Task Designer

This role focuses on developing a large-scale benchmark aimed at evaluating the capabilities of advanced AI systems in solving complex scientific and engineering challenges. As a task designer, you will create innovative computational problems that assess whether AI can effectively utilize real scientific software for research-level tasks, including running simulations, interpreting results, designing experiments, and extracting valuable insights from data.

This position goes beyond typical data-labeling jobs. You will design original, graduate-level problems based on authentic scientific workflows, rigorously test them against cutting-edge AI models, and refine them to achieve the appropriate level of difficulty.

• Create problems that necessitate proficient use of specialized scientific software, including tasks that require the AI to compute exact answers from defined setups and those that demand strategic planning to uncover hidden information.
• Engage in a testing loop with state-of-the-art AI models, refining problems until they meet the desired difficulty level.

We are particularly interested in candidates with extensive, hands-on experience in:

• Computational Bayesian Statistics and Applied Mathematics using libraries such as:
• Bayesian statistics: PyMC, PyStan, PyJAGS, CmdStanPy
• Applied mathematics and numerical PDEs: FEniCS, FEniCSx, DOLFINx, scikit-fem, FiPy, Devito, Dedalus
• Computational topology: GUDHI
• Differential algebra: DACEyPy
• Optimization: lmfit
• Experience with MCMC, Bayesian modeling, finite element or finite difference methods, mesh-based numerical modeling, computational topology, differential algebra, or other specialized Python-based math and statistics methods is advantageous.
• Experience with other specialized software in this domain will also be considered.

The ideal candidate possesses graduate-level expertise (MS or PhD preferred) in the relevant domain, with practical experience using the specified tools. You should have a proven track record of writing code with these libraries to solve real research problems, along with an understanding of their limitations and edge cases.

Additionally, strong candidates exhibit puzzle design thinking, crafting problems where the challenge stems from intelligent reasoning rather than mere computation, and where careful analysis is necessary to identify the correct approach.

• Graduate-level training in a relevant STEM field (MS, PhD, or equivalent research experience)
• Proven proficiency with at least one of the specified scientific software libraries, demonstrated through research publications, open-source contributions, or professional experience
• Strong Python programming skills for writing problem setups, oracle functions, and solution validators
• Ability to work independently and adapt problem designs based on feedback
• Comfortable working in a Linux/terminal environment with remote compute sandboxes
• Availability for at least 15, 20 hours per week

• Experience across multiple listed domains or tools
• Familiarity with benchmark or evaluation design
• Background in scientific teaching or exam/problem-set design
• Experience with computational reproducibility and containerized environments

Please note that this application includes a coding assessment as part of the evaluation process.