Research Evaluation Specialist

Role Overview

Apply advanced subject-matter expertise to develop rigorous evaluation material that helps train and assess next-generation AI systems. This remote contract role focuses on producing accurate, challenging, real-world research tasks across disciplines including finance, healthcare, STEM, engineering, and related fields. Prior AI experience is not required.

Key Responsibilities

• Create original, high-difficulty question-and-answer pairs that test AI capabilities within your discipline.
• Research, verify, and document answers using primary sources and authoritative references, with clear citations and reasoning.
• Develop questions requiring advanced reasoning, methodological nuance, or synthesis across multiple sources.
• Test questions with AI systems, identify items that are insufficiently challenging, and increase complexity while preserving accuracy.
• Write precise, unambiguous questions and defensible answers.
• Incorporate reviewer feedback and follow all project guidelines and quality standards.

Qualifications

• Completed PhD, active PhD candidacy, or equivalent research experience as a specialist, researcher, or professor in your field.
• Demonstrated scholarly research experience, deep domain expertise, and familiarity with primary literature.
• Strong research and source-triangulation skills, analytical thinking, attention to detail, and precise English writing.
• Ability to work independently and reliably in a remote engagement.
• Experience creating original, methodologically sound, challenging questions.
• AI training or evaluation experience is beneficial but not required.

Work Terms

• Remote, independent contractor engagement.
• Compensation is output-based and paid for tasks that meet project specifications. Completion time varies by experience and workflow.
• Minimum submission requirements apply, including a minimum number of tasks submitted each week.
• Selected candidates should be prepared to begin their first tasks within 24 to 48 hours after completing the required setup process.

Compensation

$40 to $90 per hour.

Application Process

Submit an application using an email address or Google account. Candidates selected for the engagement will complete the required setup before beginning task work.