Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Behavioral Health Specialist - AI Evaluation

$45 - $70 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Behavioral Health Expert, AI Safety and Model Evaluation
Type: Contract
Compensation: $45–$70/hour
Location: Remote
Commitment: 20–40 hours/week

Role Responsibilities

  • Evaluate conversations between users and AI models on topics like relationships, emotional wellbeing, and beliefs. Assess if responses are neutral, appropriate, and safe.
  • Identify patterns of concern such as excessive agreement or moralizing. Document them with clear written rationale.
  • Develop rubrics and guidelines for balanced, supportive replies based on counseling and behavioral health practices.
  • Design test scenarios to probe how models handle sensitive but non-crisis topics.
  • Collaborate with researchers to maintain consistent and well-documented evaluation standards.

Qualifications

Must-Have

  • Degree in psychology, counseling, social work, behavioral health, or equivalent professional experience.
  • 3+ years of professional experience in mental health, counseling, or social services.
  • Ability to remain neutral and nonjudgmental across diverse perspectives.
  • Familiarity with concepts like sycophancy and cognitive distortions.
  • Ability to engage reliably for 20+ hours/week during weekdays.
  • Strong written communication skills.

Preferred

  • Background in AI safety , applied ethics, or content policy.
  • Experience in relationship counseling or spiritual care.
  • Familiarity with the psychology of misinformation.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 12 days ago
Similar jobs that could be interesting for youBased on the Behavioral Health Specialist - AI Evaluation in New York, NY vacancy
  •  ...The Arizona Health Care Cost Containment System (AHCCCS) is seeking a Behavioral Health Prior Authorization Utilization Review Consultant. The role focuses on monitoring, evaluating, and approving or denying payment for medical and behavioral health services for the Fee... 
    Suggested
    Remote work

    State of Arizona

    New York, NY
    4 days ago
  • $69.79k

     ...Behavioral Health Specialist Needed Full-time, IMT Position Program/Department Description: Intensive Mobile Treatment (IMT) is an innovative...  ...social supports. Complete required assessments and evaluations as needed on a rotating schedule. Provide IMT... 
    Suggested
    Full time
    Flexible hours
    Weekend work
    Day shift
    Afternoon shift

    Bond Health

    New York, NY
    1 day ago
  • $76.72k

     ...database systems, which are used to conduct descriptive and evaluation research on arrestee characteristics, case processing and...  ...boroughs of NYC, employing over 300 employees, citywide. BEHAVIORAL HEALTH SPECIALIST INTENSIVE CASE MANAGEMENT QUEENS SUPERVISED RELEASE... 
    Suggested
    Work from home
    Flexible hours
    Night shift

    New York City Criminal Justice Agency

    New York, NY
    2 days ago
  •  ...Job Description Job Description Job Description Behavioral Health Specialist Job Title: Behavioral Health Specialist Location: [City...  ...Conduct behavioral health assessments and psychosocial evaluations. Develop, implement, and monitor individualized treatment... 
    Suggested
    Work at office
    Remote work
    Flexible hours

    Ova Technologies

    New York, NY
    20 days ago
  •  ...FHE Health in the United States seeks an experienced Utilization Review Specialist to manage clinical authorization processes, ensuring timely, accurate, and compliant...  ...optimization, compliance, and high-quality patient care across behavioral health services. #J-18808-Ljbffr
    Suggested
    Remote work

    The Florida House

    New York, NY
    4 days ago
  • $63.65k

     ...application due to a disability, contact this employer to ask for an accommodation or an alternative application process. Behavioral Health Specialist I IMT BRONX 1 Hourly Full-Time [Ben Elig] PROFESSIONAL Harlem 2090, New York, NY, US 5 days ago Requisition ID: 252... 
    Hourly pay
    Full time
    Work at office
    Remote work
    Trial period
    Monday to Friday
    Flexible hours
    Shift work
    Day shift

    Cases , Inc.

    New York, NY
    2 days ago
  •  ...Above Expectations, LLC is seeking a dedicated and motivated Behavior Specialist Consultant (BSC) to join our growing team! At Above...  ...Develop Individualized Support Strategies: Design, implement, and evaluate positive behavior intervention programs that encourage skill... 
    Weekly pay
    Temporary work
    Work at office
    Local area

    Above Expectations LLC

    New York, NY
    2 days ago
  • $36 per hour

     ...development of our employees. Position Summary: The Behavior Intervention Specialist (BIS) will assess and monitor specific challenging...  ...being collected and documented on the agency's electronic health platforms, i.e. PrecisionCare and Power BI LMSW or LCSW... 
    Local area

    Services for the UnderServed

    New York, NY
    5 days ago
  • CASES in Harlem, NY is seeking a Behavioral Health Specialist I to join a multidisciplinary team delivering client-centered treatment plans for justice-involved individuals. The role requires a bachelor’s degree and at least two years’ experience; Spanish proficiency is... 
    Work at office

    Center For Alternative

    New York, NY
    4 days ago
  • Acadia Healthcare is seeking remote Admissions Services Specialists to support our Acute Behavioral Health Facilities from coast to coast. This position is 100% remote. PRN Highlights include verifying benefits information, processing referrals, and responding to inquiries... 
    Remote job
    Relief

    Acadia Healthcare

    New York, NY
    2 days ago
  • JOB SUMMARYBased on training, experience and qualifications, the Behavioral Health Specialist may be called upon to provide one or more of the following services to children and families enrolled in the specific services. In no situation shall a Behavioral Health Specialist... 
    Minimum wage
    Temporary work
    Self employment
    Work at office
    Night shift

    Abbott House

    New York, NY
    6 days ago
  • $20 - $60 per hour

    AI Question and Answer Evaluation Specialist Create research-backed questions and answers, test AI model responses, and improve evaluation content remotely for $20 to $60 per hour equivalent, paid per task. $20-$60/hr equivalent, paid per accepted task Compensation... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • $60 per hour

    Prolific is seeking Computer Science Specialists to join our Expert Network to train and evaluate AI models. You’ll complete a quick 10- to 15-minute test and, if successful, join Prolific as a paid participant working from home with flexible hours. Researchers pay for... 
    Remote job
    Work from home
    Flexible hours

    Prolific

    New York, NY
    2 days ago
  • $30 - $50 per hour

    AI Privacy, Safety And Evaluation Specialist Test personal AI agents and review responses for privacy risks in a six-week, part-time U.S. project paying $30 to $50 per hour. Remote Hourly · $30-$50/hr $30-$50/hr Compensation Contract, part-time Engagement Remote... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • AuraOne, Inc. is seeking a Domain Contributor Evaluation Specialist for a remote contractor role in the United States. You will review AI outputs against workflows, policy, and stakeholder-fit using a structured rubric, and document the next steps for model training. Responsibilities... 
    Remote job
    For contractors
    10 hours per week

    AuraOne, Inc.

    New York, NY
    5 days ago
  • Community Access is a leading provider in New York City, delivering Integrated Mobile Treatment (IMT). The IMT Behavioral Health Specialist plays a key role within a multi-disciplinary team, engaging participants through assessment and therapeutic planning to support wellness... 

    Community Access Inc

    New York, NY
    5 days ago
  • New York City Criminal Justice Agency (CJA) in Queens, NY seeks a Behavioral Health Specialist for intensive case management within the Queens Supervised Release program. You will provide community-based support to high-risk, high-need clients and work with a multidisciplinary... 

    New York City Criminal Justice Agency

    New York, NY
    2 days ago
  • Allegheny Health Network seeks a Patient Access Coordinator for Behavioral Health in Wexford, PA. The role handles scheduling, preregistration, financial clearance, and insurance verification, ensuring accurate data and smooth patient flow. You will communicate clearly... 

    Highmark Health

    New York, NY
    3 days ago
  • Beth Israel Lahey Health, Inc. seeks a Bed Placement and Transfer Center Counselor to triage patient intake and transfers, ensuring timely...  ...of transfers to ensure seamless patient care within our behavioral services network. #J-18808-Ljbffr Beth Israel Lahey Health

    Beth Israel Lahey Health

    New York, NY
    6 days ago
  • Northeast Health Services, part of Transformations Care Network, seeks an Intake Call Center Representative to connect individuals with behavioral health services and support. You will handle screenings, guide clients to appropriate care, and document interactions with... 
    Remote job

    Transformations Care Network

    New York, NY
    4 days ago
  • $8 - $65 per hour

    Epidemiology Specialist - Freelance AI Trainer Project United States of America Are you a master of...  ...world‑class education, accelerate public health insights, and support evidence-based...  ...to our prompt engineering and evaluation metrics. A PhD degree in epidemiology... 
    Remote job
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start

    Meridial

    New York, NY
    2 days ago
  • $21 - $35 per hour

    Crossing Hurdles is seeking an Image Evaluation Expert (Visual Arts / Photography) on a remote contract basis. You will upload photos, compare AI-generated images with real photographs, and assess lighting, perspective, and textures to ensure visual accuracy. You will... 
    Remote job
    Hourly pay
    Contract work

    Crossing Hurdles

    New York, NY
    2 days ago
  • CASES in New York City seeks a Behavioral Health Specialist I to join a multidisciplinary team serving justice-involved individuals with serious mental illness. The role emphasizes integrated treatment planning, crisis intervention, medication management, and community... 
    Flexible hours

    CASES

    New York, NY
    2 days ago
  • The Educational Alliance Inc. seeks a Clinical Assistant/Behavioral Health Screener in New York, NY. This full-time role combines front desk duties, initial behavioral health screenings, intake coordination, and insurance verification to ensure timely access to services... 
    Full time

    The Educational Alliance Inc.

    New York, NY
    3 days ago
  • Softechub is seeking an Image Evaluation Expert to remotely support our AI-driven image assessment projects in the United States. You will rate photos, provide detailed feedback, and help train next-generation AI systems with high-quality real-world input. The role emphasizes... 
    Remote job

    Softechub

    New York, NY
    2 days ago
  • $25 - $35 per hour

     ...passionate about helping children and families thrive? Do you have experience working in mental/behavioral health? We are looking for dedicated Behavioral Health Specialists to provide individualized support services to youth and families in the foster care system who are... 
    Part time

    Greystone-Programs,-Inc.

    New York, NY
    4 days ago
  • Center for Alternative Sentencing & Employment Services (CASES) in New York, NY seeks a Behavioral Health Specialist I to support justice-involved individuals with behavioral health needs. The role involves assessment, treatment planning, and linkages to medical care,... 
    Full time

    Center for Alternative Sentencing & Employment Services

    New York, NY
    4 days ago
  • Educational Alliance’s Center for Recovery and Wellness seeks a Clinical Assistant/Behavioral Health Screener to serve as the first point of contact for behavioral health services. The role combines front desk duties, initial screening, intake coordination, and insurance... 

    Educational Alliance

    New York, NY
    6 days ago
  • Crossing Hurdles is seeking an Image Evaluation Expert for a remote contract role in the United States. You will upload photographs, compare real images with AI outputs, and assess lighting, perspective, and textures to ensure accuracy. You will provide clear, evidence-... 
    Remote job
    Contract work
    10 hours per week

    Crossing Hurdles

    New York, NY
    5 days ago
  • $63.65k

     ...Job Description Job Description Job Title:   Behavioral Health Specialist I   Job Summary: CASES is a New York City agency working at the forefront of the intersection between behavioral health and involvement in the criminal justice system. Our mission... 
    Full time
    Work at office
    Remote work
    Trial period
    Monday to Friday
    Flexible hours
    Shift work
    Day shift

    CASES

    New York, NY
    11 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Behavioral Health Specialist - AI Evaluation. Be the first to apply!