Evaluation Specialist for AI Model Evaluation

Role Overview

Help train next-generation AI systems by creating rigorous, real-world evaluations that test how advanced models learn, reason, and perform. This remote contractor opportunity is open to recent graduates, advanced-degree holders, and professionals with strong research and writing capabilities. Prior AI experience is not required.

Key Responsibilities

• Develop original, challenging question-and-answer pairs across diverse subjects.
• Conduct in-depth research and triangulate multiple sources to produce accurate, comprehensive, well-documented answers.
• Create multi-step questions that require synthesis and analytical reasoning rather than simple single-source lookups.
• Test questions with AI models, analyze the results, and adjust difficulty as needed.
• Document research methods with clear citations and logical explanations.
• Revise content in response to reviewer feedback while following project guidelines and quality standards.

Qualifications

• Demonstrated ability to independently research and critically assess information from multiple sources.
• Excellent attention to detail, analytical thinking, and precise English writing. English fluency is required, though it need not be your first language.
• Ability to write nuanced, well-structured questions that assess deep understanding.
• Strong self-direction, reliability, and accountability in remote independent work.
• Interest in advanced AI and evaluating current model capabilities.
• Experience with AI training, evaluation, or content creation is helpful but not required.

Work Terms

• Remote contract engagement.
• Work is paid by qualifying task output, and completion time may vary based on experience and workflow.
• Minimum submission requirements apply, including a minimum number of tasks submitted each week.
• Roles are commonly filled within 48 hours. Selected candidates should be prepared to begin their first tasks within 24 to 48 hours after completing onboarding.

Compensation

Compensation is listed at $20 to $60 per hour, with payment determined on a per-task basis for work that meets project specifications.

Application Process

Submit an application using an email address or Google account. Candidates who advance complete onboarding before beginning project tasks. Questions can be reviewed through the available FAQs.