Psychiatrist for Large Language Model Evaluation

Apply your psychiatric expertise to evaluate how advanced AI systems reason through complex clinical cases. You will create challenging diagnostic and clinical-reasoning scenarios, identify gaps in model performance, and provide authoritative answers for a project supporting a leading AI research lab. Key Responsibilities

• Evaluate AI reasoning on advanced psychiatric cases.
• Develop gold-standard answers supported by clear clinical reasoning.
• Focus on edge cases, nuanced differential diagnoses, and risk assessment.
• Collaborate directly with research teams.
• Provide feedback on recurring failure modes in model reasoning.

• Completed an ACGME, AOA, or nationally equivalent psychiatry residency. In-training candidates are not eligible.
• MD or DO degree, including international medical degrees.
• Active board certification in psychiatry through ABPN, FRCPC, MRCPsych, FRANZCP, or a recognized national equivalent.
• Active, unrestricted medical license with no suspension, revocation, or open disciplinary action.
• At least 3 years of independent practice after residency, either currently or within the past 2 years. Training years do not count.
• Practice in an accredited or government-licensed hospital, academic center, community clinic, private practice, or telehealth setting.
• Psychiatrists from all practice settings may apply, with particular interest in subspecialty expertise.

• Currently provide at least 10 hours per week of direct patient clinical care. This is scored as a preference, not a required threshold.
• Psychiatry-focused expertise in Addiction, Child and Adolescent, Forensic, Geriatric, or Consultation-Liaison Psychiatry, demonstrated through fellowship training, subspecialty certification, or at least 2 years of practice.
• OSCE development or examiner experience. Candidate participation alone does not qualify.
• Current faculty appointment in a psychiatry department, or one held within the past 2 years.

• Remote, hourly engagement.
• Applicants must be based in the United States, United Kingdom, Europe, or Australia. United States-based applicants are preferred.

• $150 per hour.

• Board-eligible psychiatrists and candidates with lapsed board certification are not eligible.

• Selected applicants complete a brief 20 to 30 minute interview focused on clinical background and subspecialty expertise.