Medical Student, Resident, or Physician for AI Model Evaluation

Apply clinical expertise to help train and evaluate next-generation AI systems. You will create rigorous medical evaluations that test nuanced reasoning, using real-world medical knowledge to improve how models learn, reason, and perform. No prior AI experience is required.

• Create original, high-difficulty medical question-and-answer pairs covering diagnosis, mechanisms, pharmacology, guidelines, and clinical judgment.
• Research, verify, and document answers with rationale and citations from primary literature, clinical guidelines, and other authoritative references.
• Develop questions that require synthesis and genuine clinical reasoning rather than simple fact recall.
• Evaluate AI responses, revise questions that are too straightforward, and increase complexity while maintaining clinical accuracy.
• Write clear, precise, defensible questions and answers that avoid ambiguity.
• Incorporate reviewer feedback and follow evolving project standards and guidelines.

• Current medical training or clinical practice as a student, resident, or physician, or equivalent biomedical expertise.
• Ability to locate, interpret, triangulate, and synthesize primary literature and clinical guidelines.
• Strong analytical thinking, exceptional attention to detail, and precise written English. Native fluency is not required.
• Ability to create engaging, well-structured, clinically nuanced questions.
• Self-direction and reliability when managing independent remote work.
• Experience with medical question writing, peer review, or evaluation projects is advantageous.
• Familiarity with AI or machine-learning systems and model evaluation is helpful but not required.

• Remote contract engagement.
• Compensation is output-based and paid per task that meets project specifications. Completion time may vary based on experience and workflow.
• Minimum submission requirements apply, including a required minimum number of tasks each week.
• Roles are typically filled within 48 hours. Selected candidates should be prepared to begin their first tasks within 24 to 48 hours after completing onboarding.

$40 to $90 per hour.