Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Clinical Mental Health Expert — AI Conversation Evaluation

Part-time

Mercor

Mercor is seeking Clinical Mental Health Experts to support an AI research initiative focused on evaluating the realism and quality of simulated clinical conversations. This is a short-term, fully remote evaluation task designed for mental health professionals who can apply their clinical experience and professional judgment to assess realistic conversational scenarios. The task will take up to 30 minutes to complete.

What You’ll Do

Review pairs of simulated conversations involving clinical mental health scenarios.

Evaluate the conversations based on their realism and authenticity.

Determine which conversation more closely reflects a realistic clinical interaction.

Apply your clinical experience and professional judgment when making evaluations.

Complete the evaluation independently through an online interface.

Ideal Background

We welcome professionals with a relevant degree and clinical mental health expertise. Suitable backgrounds may include:

Clinical Psychology

Counselling Psychology

Mental Health Counselling

Clinical Social Work

Psychiatry

Psychotherapy

Behavioural Health

Marriage and Family Therapy

Other closely related clinical mental health disciplines

Candidates should have:

A relevant degree or professional qualification in a mental health-related field.

Practical experience working in a clinical mental health or behavioural health setting.

Familiarity with real-world conversations between mental health professionals and patients or clients.

Strong professional judgment and the ability to identify whether clinical interactions feel realistic and authentic.

Strong attention to detail and the ability to make independent qualitative assessments.

Why Join

Apply your clinical expertise to the evaluation of advanced AI systems.

Contribute to research aimed at improving the realism and quality of AI-generated interactions.

Be considered for future AI evaluation and research opportunities requiring clinical mental health expertise.

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Clinical Mental Health Expert — AI Conversation Evaluation in San Francisco, CA vacancy
  • $150 per hour

     ...technical talent with leading AI research labs. Headquartered in...  ...Clinician (MD/DO) - HLS Expert Pool Type: Contract...  ...grading criteria by breaking clinical questions into discrete checkable...  ...model response grading. Evaluate clinical dialogues for accuracy... 
    Suggested
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    6 days ago
  • $45 - $70 per hour

     ...Science Help a leading AI lab ensure its...  ...seeking behavioral health experts to help evaluate and improve how its...  ...sensitive, everyday conversations. People increasingly...  ...professional experience in mental health, counseling,...  ...overstepping into clinical or directive advice,... 
    Suggested
    Contract work
    Part time
    Weekday work

    Mercor

    San Francisco, CA
    1 day ago
  • $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...drug-development programs. Interpret clinical trial endpoints and judge whether results...  ...asynchronously to meet deadlines while improving AI evaluation . Qualifications Must-Have US-... 
    Suggested
    Hourly pay
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    23 days ago
  • $70 - $110 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ..., and Jack Dorsey . Position: Clinical Medicine Domain Expert Type: Contract Compensation:...  .../week Role Responsibilities Evaluate the quality of clinical knowledge work... 
    Suggested
    Hourly pay
    Weekly pay
    Contract work
    For contractors
    Summer work

    Mercor

    San Francisco, CA
    23 days ago
  •  ...Licensed psychologists to review how AI models handle clinical psychology scenarios, and help...  ...realistic cases. AfterQuery builds the expert evaluation data that frontier AI labs use to...  ...and improve how their models handle mental health. We're bringing on licensed psychologists... 
    Suggested
    Weekly pay
    Contract work
    Temporary work
    For contractors
    Remote work

    AfterQuery

    San Francisco, CA
    1 day ago
  • $80 - $150 per hour

     ...technical talent with leading AI research labs....  ...Position: Child & Adolescent Mental Health Clinical Advisor (AI Safety Benchmark...  ...clinician-informed evaluation rubrics for AI chatbot responses...  ...a subset of AI-generated conversations and provide expert judgment to calibrate... 
    Contract work
    Traineeship
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $60 per hour

    Prolific seeks Chemistry Experts and Chemical Engineers to join our Expert Network and evaluate AI models using your chemical expertise. Successful candidates will be invited to assess and review AI-generated chemistry tasks and ensure their accuracy. Compensation can reach... 
    Remote job
    Hourly pay
    Flexible hours

    Prolific

    San Francisco, CA
    2 days ago
  • $70 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  .... Position: Mathematics PhD Coding Experts Type: Contract Compensation:...  ...resume and application form. A 25-minute conversational interview covering your background, experience... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    23 days ago
  • $60 - $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $90 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...genuinely correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use of... 
    Contract work
    Summer work
    Local area
    Remote work

    Mercor

    San Francisco, CA
    2 days ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Dorsey . Position: Software Engineer Expert Type: Contract Compensation: $...  ...Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    24 days ago
  •  ...Mercor, we believe the safest AI is the one that’s already...  ...for this project - human data experts who probe AI models with adversarial...  ...What You’ll Do Red team conversational AI models and agents:...  ...strengthen customer AI systems Evaluation coverage expands: more scenarios... 
    Part time
    Remote work

    Mercor

    San Francisco, CA
    1 day ago
  •  ...psychiatrists to review how AI models handle psychiatric cases...  .... AfterQuery builds the expert evaluation data that frontier AI labs use...  ...how their models handle mental health. We're bringing on licensed...  ...contributors, help write realistic clinical vignettes and reference... 
    Weekly pay
    Contract work
    For contractors
    Remote work

    AfterQuery

    San Francisco, CA
    10 days ago
  •  ...Create and evaluate board game tasks to enhance AI reasoning skills. About Turing Turing is one of the world’s fastest-growing AI companies, accelerating...  ...Role Overview We are seeking Board Game Reasoning Experts to help train and evaluate next-generation AI models.... 
    Contract work
    Part time
    For contractors
    Freelance
    Remote work

    Turing

    San Francisco, CA
    13 days ago
  • $50 - $150 per hour

     ...hours / week Experience: 3 - 5 years Location: United States | Canada + 6 more We're hiring PhD-level experts to help train and evaluate advanced AI models. This is remote, flexible contract work where your academic expertise directly shapes how cutting-edge... 
    Contract work
    Temporary work
    Remote work
    Flexible hours

    Braintrust

    San Francisco, CA
    13 days ago
  • $164.5k - $219k

     ...Information Job Title Expert Senior Manager, Data Sciences...  ...opportunities to work with major AI ecosystem partners through...  ...team applies to prompt design, evaluation and output quality...  ...work, and supporting commercial conversations with clientsLead and develop... 
    Permanent employment
    Full time
    Apprenticeship
    Local area
    Home office
    3 days per week

    Bain & Company

    San Francisco, CA
    7 days ago
  • $105 per hour

     ...technical talent with leading AI research labs. Headquartered in...  ...workflows for medical and clinical services across multiple payer...  ...or payer-specific criteria. Evaluate AI-generated prior authorisation...  ...office, hospital, or health plan prior authorisation operations... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    22 days ago
  •  ...LCSWs to build the clinical social work scenarios frontier AI labs use to test model judgment and safety. AfterQuery builds the expert evaluation data that frontier AI labs use to measure and improve their models. We're hiring licensed clinical social workers as contractors... 
    Weekly pay
    Contract work
    Temporary work
    For contractors
    Remote work
    10 hours per week

    AfterQuery

    San Francisco, CA
    1 day ago
  • $8 - $65 per hour

     ...Kaqchikel (Mayan) Language Expert - Freelance AI Trainer Project at Meridial...  ...On a typical day, you will converse with the model in Kaqchikel...  ...our prompt engineering and evaluation metrics. A Master's degree...  ...sponsored benefits such as health insurance and PTO do not... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Freelance
    Immediate start
    Remote work

    Invisible Technologies

    San Francisco, CA
    1 day ago
  •  ...technical talent with leading AI research labs. Headquartered in...  ...and scenarios focusing on clinical trials to reveal weaknesses in AI model responses. Evaluate AI-generated clinical reasoning...  ...Provide clear well-structured expert responses and chain-of-thought... 
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    7 days ago
  • $293.5k

    General Information Job Title Expert Senior Manager, AI Engineering Job ID 104335 Work...  ..., model experimentation, and evaluation design to production system deploymentOversee...  ..., context engineering, and conversation designStrong expertise in evaluation... 
    Permanent employment
    Full time
    Apprenticeship
    Work at office
    Local area
    Work from home
    Home office
    3 days per week

    Bain & Company

    San Francisco, CA
    4 days ago
  • $39 per hour

     ...technical talent with leading AI research labs. Headquartered...  .... Position: Music Audio Expert - Swedish Type: Contract...  ...Role Responsibilities Evaluate AI model output lyrics , voice...  ...form Complete a 25-minute conversational interview on your background,... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  • $50 - $60 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Summers , and Jack Dorsey . Position: Expert Interviewer Type: Contract Compensation...  ...technical depth, communication, and AI-evaluation fit using a provided rubric. Confirm... 
    Contract work
    Summer work
    Remote work
    Weekday work

    Mercor

    San Francisco, CA
    22 days ago
  • $50 - $70 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Jack Dorsey . Position: Generalist Expert (US/Canada) Type: Contract Compensation...  ...Role Responsibilities Review and evaluate documents, slides, spreadsheets, and similar... 
    Contract work
    Summer work
    Remote work
    Flexible hours

    Mercor

    San Francisco, CA
    26 days ago
  • $55 - $65 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Role Responsibilities Review and evaluate AI-generated clinical outputs based on nursing flowsheet documentation...  ...AI training datasets . Provide expert feedback on nursing assessments and... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $24 - $35 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Jack Dorsey . Position: AI Safety Experts — English & Thai Type: Contract...  ...Remote Role Responsibilities Red team conversational AI models and agents. Focus on jailbreaks... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    16 days ago
  • $2,200 per month

     ...Build real-world tasks that test how well AI agents handle database engineering....  ...retries, and regressions where relevant. Evaluate performance measures such as latency, throughput...  ...Paid task after selection Selected experts will complete one task, with... 
    Temporary work
    Immediate start

    Mercor

    San Francisco, CA
    1 day ago
  •  ...Clinical Specialist Are you looking for a unique and exciting opportunity to join a growing division at a blue chip medical device manufacturer? We are currently seeking a Clinical Specialist to join our team and help spearhead clinical trials for a revolutionary new... 

    Exelare

    San Francisco, CA
    4 days ago
  • $60 per hour

    Prolific is seeking Accountant Subject Matter Experts to join their Expert Network to provide professional insights in AI training. Responsibilities include evaluating financial documents, providing AI model guidance, and ensuring data integrity. The ideal candidates will... 
    Remote job
    Hourly pay
    Work from home
    Flexible hours

    Prolific

    San Francisco, CA
    4 days ago
  • $20 - $22 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Jack Dorsey . Position: AI Safety Experts — English & Assamese Type: Contract...  ...Remote Role Responsibilities Red team conversational AI models and agents, focusing on... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Clinical Mental Health Expert — AI Conversation Evaluation. Be the first to apply!