STEM Researcher for AI Model Evaluation

Help build next-generation agentic evaluation benchmarks for frontier AI models by turning rigorous scientific practice into challenging, multi-step evaluation tasks. You will design experiments, implement and run them in code, analyze results, and write careful conclusions that expose where state-of-the-art models fail. Tasks are designed to require one to two days of focused work and combine study design, computational implementation, and clear written criteria for success.

• Design tasks that operationalize experimental design, hypothesis testing, and rigorous evaluation into multi-step challenges that current models do not reliably solve.
• Author and execute solutions in Python and notebook environments, documenting methods and expected results at the level of a careful researcher.
• Define scoring and success criteria, specifying what distinguishes sound scientific reasoning from superficially plausible answers.
• Evaluate model outputs against your tasks, identify methodological and reasoning errors, and flag mistakes a working researcher would notice immediately.
• Collaborate with researchers and subject-matter experts to align evaluations, share findings, and keep assessments consistent.
• Produce reproducible code, analyses, and written summaries for each task you create.

• MSc or PhD in a STEM field, or in a computational social science or humanities discipline, or equivalent hands-on experience in a research-heavy role involving coding and data analysis.
• At least 1 year of experience in an active research position, in academia, industry, or national labs.
• Significant computational research experience, including Python-based analysis, simulation, modeling, or building data pipelines.
• Strong grounding in experimental design, hypothesis testing, and rigorous evaluation.
• Familiarity with Git, development IDEs, and notebook environments such as Jupyter or Colab.
• Preferred: prior experience with AI model training, model evaluation, or authoring benchmarks and tasks.
• Highly detail oriented, creative in task design, excellent written communication, and able to work independently on ambiguous, open-ended problems.
• Ability to commit approximately 35 hours per week.

• This is a W-2 employment role with Cincinnatus LLC, intended as a structured, role-based position rather than a freelance or project-only engagement.
• Placement will be with a leading AI research team as part of their extended workforce, working closely within client teams and standard enterprise workflows.
• The position is fully remote within the United States.
• Typical time commitment is approximately 35 hours per week. Individual tasks generally take one to two days of continuous focused effort.

• Hourly pay range: $60.00 to $90.00 per hour.
• Compensation is paid through Cincinnatus LLC as the employer of record.

• Because this is W-2 employment with a U.S. employer, candidates must be able to be employed by a United States-based company while working remotely within the United States.
• Cincinnatus LLC handles employment administration, including offers, onboarding, payroll, benefits, and compliance for this role.
• Opportunities may be listed or discovered via partner platforms, however hiring, onboarding, and employment administration are managed by Cincinnatus LLC.
• Cincinnatus is an Equal Employment Opportunity employer and will provide reasonable accommodations for qualified individuals with disabilities during the application process.