Clinical Mental Health Expert — AI Conversation Evaluation
Mercor
Mercor is seeking Clinical Mental Health Experts to support an AI research initiative focused on evaluating the realism and quality of simulated clinical conversations. This is a short-term, fully remote evaluation task designed for mental health professionals who can apply their clinical experience and professional judgment to assess realistic conversational scenarios. The task will take up to 30 minutes to complete.
What You’ll Do
Review pairs of simulated conversations involving clinical mental health scenarios.
Evaluate the conversations based on their realism and authenticity.
Determine which conversation more closely reflects a realistic clinical interaction.
Apply your clinical experience and professional judgment when making evaluations.
Complete the evaluation independently through an online interface.
Ideal Background
We welcome professionals with a relevant degree and clinical mental health expertise. Suitable backgrounds may include:
Clinical Psychology
Counselling Psychology
Mental Health Counselling
Clinical Social Work
Psychiatry
Psychotherapy
Behavioural Health
Marriage and Family Therapy
Other closely related clinical mental health disciplines
Candidates should have:
A relevant degree or professional qualification in a mental health-related field.
Practical experience working in a clinical mental health or behavioural health setting.
Familiarity with real-world conversations between mental health professionals and patients or clients.
Strong professional judgment and the ability to identify whether clinical interactions feel realistic and authentic.
Strong attention to detail and the ability to make independent qualitative assessments.
Why Join
Apply your clinical expertise to the evaluation of advanced AI systems.
Contribute to research aimed at improving the realism and quality of AI-generated interactions.
Be considered for future AI evaluation and research opportunities requiring clinical mental health expertise.
We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.
$150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Clinician (MD/DO) - HLS Expert Pool Type: Contract... ...grading criteria by breaking clinical questions into discrete checkable... ...model response grading. Evaluate clinical dialogues for accuracy...SuggestedFull timeContract workSummer workRemote work$45 - $70 per hour
...Science Help a leading AI lab ensure its... ...seeking behavioral health experts to help evaluate and improve how its... ...sensitive, everyday conversations. People increasingly... ...professional experience in mental health, counseling,... ...overstepping into clinical or directive advice,...SuggestedContract workPart timeWeekday work$150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...drug-development programs. Interpret clinical trial endpoints and judge whether results... ...asynchronously to meet deadlines while improving AI evaluation . Qualifications Must-Have US-...SuggestedHourly payContract workFor contractorsSummer workRemote work$70 - $110 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: Clinical Medicine Domain Expert Type: Contract Compensation:... .../week Role Responsibilities Evaluate the quality of clinical knowledge work...SuggestedHourly payWeekly payContract workFor contractorsSummer work- ...Licensed psychologists to review how AI models handle clinical psychology scenarios, and help... ...realistic cases. AfterQuery builds the expert evaluation data that frontier AI labs use to... ...and improve how their models handle mental health. We're bringing on licensed psychologists...SuggestedWeekly payContract workTemporary workFor contractorsRemote work
$80 - $150 per hour
...technical talent with leading AI research labs.... ...Position: Child & Adolescent Mental Health Clinical Advisor (AI Safety Benchmark... ...clinician-informed evaluation rubrics for AI chatbot responses... ...a subset of AI-generated conversations and provide expert judgment to calibrate...Contract workTraineeshipSummer workRemote work$60 per hour
Prolific seeks Chemistry Experts and Chemical Engineers to join our Expert Network and evaluate AI models using your chemical expertise. Successful candidates will be invited to assess and review AI-generated chemistry tasks and ensure their accuracy. Compensation can reach...Remote jobHourly payFlexible hours$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Mathematics PhD Coding Experts Type: Contract Compensation:... ...resume and application form. A 25-minute conversational interview covering your background, experience...Contract workSummer workImmediate startRemote work$60 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance...Contract workSummer workRemote work$90 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...genuinely correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use of...Contract workSummer workLocal areaRemote work$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: Software Engineer Expert Type: Contract Compensation: $... ...Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...Contract workSummer workWork at officeRemote work- ...Mercor, we believe the safest AI is the one that’s already... ...for this project - human data experts who probe AI models with adversarial... ...What You’ll Do Red team conversational AI models and agents:... ...strengthen customer AI systems Evaluation coverage expands: more scenarios...Part timeRemote work
- ...psychiatrists to review how AI models handle psychiatric cases... .... AfterQuery builds the expert evaluation data that frontier AI labs use... ...how their models handle mental health. We're bringing on licensed... ...contributors, help write realistic clinical vignettes and reference...Weekly payContract workFor contractorsRemote work
- ...Create and evaluate board game tasks to enhance AI reasoning skills. About Turing Turing is one of the world’s fastest-growing AI companies, accelerating... ...Role Overview We are seeking Board Game Reasoning Experts to help train and evaluate next-generation AI models....Contract workPart timeFor contractorsFreelanceRemote work
$50 - $150 per hour
...hours / week Experience: 3 - 5 years Location: United States | Canada + 6 more We're hiring PhD-level experts to help train and evaluate advanced AI models. This is remote, flexible contract work where your academic expertise directly shapes how cutting-edge...Contract workTemporary workRemote workFlexible hours$164.5k - $219k
...Information Job Title Expert Senior Manager, Data Sciences... ...opportunities to work with major AI ecosystem partners through... ...team applies to prompt design, evaluation and output quality... ...work, and supporting commercial conversations with clientsLead and develop...Permanent employmentFull timeApprenticeshipLocal areaHome office3 days per week$105 per hour
...technical talent with leading AI research labs. Headquartered in... ...workflows for medical and clinical services across multiple payer... ...or payer-specific criteria. Evaluate AI-generated prior authorisation... ...office, hospital, or health plan prior authorisation operations...Contract workSummer workWork at officeRemote work- ...LCSWs to build the clinical social work scenarios frontier AI labs use to test model judgment and safety. AfterQuery builds the expert evaluation data that frontier AI labs use to measure and improve their models. We're hiring licensed clinical social workers as contractors...Weekly payContract workTemporary workFor contractorsRemote work10 hours per week
$8 - $65 per hour
...Kaqchikel (Mayan) Language Expert - Freelance AI Trainer Project at Meridial... ...On a typical day, you will converse with the model in Kaqchikel... ...our prompt engineering and evaluation metrics. A Master's degree... ...sponsored benefits such as health insurance and PTO do not...Hourly payContract workPart timeFor contractorsFreelanceImmediate startRemote work- ...technical talent with leading AI research labs. Headquartered in... ...and scenarios focusing on clinical trials to reveal weaknesses in AI model responses. Evaluate AI-generated clinical reasoning... ...Provide clear well-structured expert responses and chain-of-thought...Full timeContract workSummer workRemote work
$293.5k
General Information Job Title Expert Senior Manager, AI Engineering Job ID 104335 Work... ..., model experimentation, and evaluation design to production system deploymentOversee... ..., context engineering, and conversation designStrong expertise in evaluation...Permanent employmentFull timeApprenticeshipWork at officeLocal areaWork from homeHome office3 days per week$39 per hour
...technical talent with leading AI research labs. Headquartered... .... Position: Music Audio Expert - Swedish Type: Contract... ...Role Responsibilities Evaluate AI model output lyrics , voice... ...form Complete a 25-minute conversational interview on your background,...Contract workSummer workImmediate startRemote work$50 - $60 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Expert Interviewer Type: Contract Compensation... ...technical depth, communication, and AI-evaluation fit using a provided rubric. Confirm...Contract workSummer workRemote workWeekday work$50 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract Compensation... ...Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar...Contract workSummer workRemote workFlexible hours$55 - $65 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Role Responsibilities Review and evaluate AI-generated clinical outputs based on nursing flowsheet documentation... ...AI training datasets . Provide expert feedback on nursing assessments and...Contract workSummer workRemote work$24 - $35 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: AI Safety Experts — English & Thai Type: Contract... ...Remote Role Responsibilities Red team conversational AI models and agents. Focus on jailbreaks...Contract workSummer workRemote work$2,200 per month
...Build real-world tasks that test how well AI agents handle database engineering.... ...retries, and regressions where relevant. Evaluate performance measures such as latency, throughput... ...Paid task after selection Selected experts will complete one task, with...Temporary workImmediate start- ...Clinical Specialist Are you looking for a unique and exciting opportunity to join a growing division at a blue chip medical device manufacturer? We are currently seeking a Clinical Specialist to join our team and help spearhead clinical trials for a revolutionary new...
$60 per hour
Prolific is seeking Accountant Subject Matter Experts to join their Expert Network to provide professional insights in AI training. Responsibilities include evaluating financial documents, providing AI model guidance, and ensuring data integrity. The ideal candidates will...Remote jobHourly payWork from homeFlexible hours$20 - $22 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: AI Safety Experts — English & Assamese Type: Contract... ...Remote Role Responsibilities Red team conversational AI models and agents, focusing on...Contract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Clinical Mental Health Expert — AI Conversation Evaluation. Be the first to apply!
- fulfillment expert San Francisco, CA
- technology expert San Francisco, CA
- subject matter expert San Francisco, CA
- guest service support expert San Francisco, CA
- entry level mental health position San Francisco, CA
- bachelors mental health San Francisco, CA
- mental health caregiver San Francisco, CA
- mental health clinician San Francisco, CA
- mental health work from home San Francisco, CA
- mental health assessment San Francisco, CA







