Mathematician for AI Model Evaluation

This role offers a unique opportunity to engage with advanced mathematical reasoning and problem-solving in the context of improving and evaluating large language models. You will apply your expertise to design challenging mathematics problems, develop computational solutions, and contribute to the evolution of AI systems. Key Responsibilities:

• Design original and challenging mathematics problems that test the reasoning limits of large language models, particularly in multi-step, abstract, and proof-based settings.
• Solve problems independently and write detailed, logically structured solutions with clear justifications.
• Review model-generated solutions, identify mathematical errors or missing arguments, and provide precise feedback, annotations, and corrections.
• Contribute to defining new evaluation benchmarks based on Mathematics curricula spanning from early undergraduate to PhD-level topics.
• Develop and validate Python-based solutions for computational tasks using approved scientific libraries.
• Work on theorem-prover tasks using Lean, including translating mathematical problems and proofs into formal language, verifying that formal proofs compile correctly.

• Strong analytical skills with good research capabilities.
• Ability to provide constructive feedback and detailed annotations.
• Creative and lateral thinking abilities.
• Excellent structured communication and collaboration skills in a remote setting.
• Self-motivated and able to work independently in a remote environment.
• Must have a desktop/laptop setup with a reliable internet connection.

• Candidates pursuing a Master’s/Ph.D./Postdoctoral degree in Mathematics, Applied Mathematics, Statistics, or a related field are encouraged to apply.
• Ability to analyze and solve complex math problems with a structured and logical approach.
• Ability to explain math concepts clearly using simple language, visuals, and examples.

This is a fully remote, contract position.

Compensation details will be discussed during the interview process.

Open to candidates with the required educational background and skills.

• Work in a fully remote environment.
• Opportunity to work on cutting-edge AI projects with leading LLM companies.
• Potential for contract extension based on performance and project needs.