Behavioral Health Specialist - AI Evaluation
$45 - $70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Behavioral Health Expert, AI Safety and Model Evaluation
Type: Contract
Compensation: $45–$70/hour
Location: Remote
Commitment: 20–40 hours/week
Role Responsibilities
- Evaluate conversations between users and AI models on topics like relationships, emotional wellbeing, and beliefs. Assess if responses are neutral, appropriate, and safe.
- Identify patterns of concern such as excessive agreement or moralizing. Document them with clear written rationale.
- Develop rubrics and guidelines for balanced, supportive replies based on counseling and behavioral health practices.
- Design test scenarios to probe how models handle sensitive but non-crisis topics.
- Collaborate with researchers to maintain consistent and well-documented evaluation standards.
Qualifications
Must-Have
- Degree in psychology, counseling, social work, behavioral health, or equivalent professional experience.
- 3+ years of professional experience in mental health, counseling, or social services.
- Ability to remain neutral and nonjudgmental across diverse perspectives.
- Familiarity with concepts like sycophancy and cognitive distortions.
- Ability to engage reliably for 20+ hours/week during weekdays.
- Strong written communication skills.
Preferred
- Background in AI safety , applied ethics, or content policy.
- Experience in relationship counseling or spiritual care.
- Familiarity with the psychology of misinformation.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
- ...The Arizona Health Care Cost Containment System (AHCCCS) is seeking a Behavioral Health Prior Authorization Utilization Review Consultant. The role focuses on monitoring, evaluating, and approving or denying payment for medical and behavioral health services for the Fee...SuggestedRemote work
$69.79k
...Behavioral Health Specialist Needed Full-time, IMT Position Program/Department Description: Intensive Mobile Treatment (IMT) is an innovative... ...social supports. Complete required assessments and evaluations as needed on a rotating schedule. Provide IMT...SuggestedFull timeFlexible hoursWeekend workDay shiftAfternoon shift$76.72k
...database systems, which are used to conduct descriptive and evaluation research on arrestee characteristics, case processing and... ...boroughs of NYC, employing over 300 employees, citywide. BEHAVIORAL HEALTH SPECIALIST INTENSIVE CASE MANAGEMENT QUEENS SUPERVISED RELEASE...SuggestedWork from homeFlexible hoursNight shift- ...Job Description Job Description Job Description Behavioral Health Specialist Job Title: Behavioral Health Specialist Location: [City... ...Conduct behavioral health assessments and psychosocial evaluations. Develop, implement, and monitor individualized treatment...SuggestedWork at officeRemote workFlexible hours
- ...FHE Health in the United States seeks an experienced Utilization Review Specialist to manage clinical authorization processes, ensuring timely, accurate, and compliant... ...optimization, compliance, and high-quality patient care across behavioral health services. #J-18808-LjbffrSuggestedRemote work
$63.65k
...application due to a disability, contact this employer to ask for an accommodation or an alternative application process. Behavioral Health Specialist I IMT BRONX 1 Hourly Full-Time [Ben Elig] PROFESSIONAL Harlem 2090, New York, NY, US 5 days ago Requisition ID: 252...Hourly payFull timeWork at officeRemote workTrial periodMonday to FridayFlexible hoursShift workDay shift- ...Above Expectations, LLC is seeking a dedicated and motivated Behavior Specialist Consultant (BSC) to join our growing team! At Above... ...Develop Individualized Support Strategies: Design, implement, and evaluate positive behavior intervention programs that encourage skill...Weekly payTemporary workWork at officeLocal area
$36 per hour
...development of our employees. Position Summary: The Behavior Intervention Specialist (BIS) will assess and monitor specific challenging... ...being collected and documented on the agency's electronic health platforms, i.e. PrecisionCare and Power BI LMSW or LCSW...Local area- CASES in Harlem, NY is seeking a Behavioral Health Specialist I to join a multidisciplinary team delivering client-centered treatment plans for justice-involved individuals. The role requires a bachelor’s degree and at least two years’ experience; Spanish proficiency is...Work at office
- Acadia Healthcare is seeking remote Admissions Services Specialists to support our Acute Behavioral Health Facilities from coast to coast. This position is 100% remote. PRN Highlights include verifying benefits information, processing referrals, and responding to inquiries...Remote jobRelief
- JOB SUMMARYBased on training, experience and qualifications, the Behavioral Health Specialist may be called upon to provide one or more of the following services to children and families enrolled in the specific services. In no situation shall a Behavioral Health Specialist...Minimum wageTemporary workSelf employmentWork at officeNight shift
$20 - $60 per hour
AI Question and Answer Evaluation Specialist Create research-backed questions and answers, test AI model responses, and improve evaluation content remotely for $20 to $60 per hour equivalent, paid per task. $20-$60/hr equivalent, paid per accepted task Compensation...Hourly payContract workPart timeFor contractorsRemote work$60 per hour
Prolific is seeking Computer Science Specialists to join our Expert Network to train and evaluate AI models. You’ll complete a quick 10- to 15-minute test and, if successful, join Prolific as a paid participant working from home with flexible hours. Researchers pay for...Remote jobWork from homeFlexible hours$30 - $50 per hour
AI Privacy, Safety And Evaluation Specialist Test personal AI agents and review responses for privacy risks in a six-week, part-time U.S. project paying $30 to $50 per hour. Remote Hourly · $30-$50/hr $30-$50/hr Compensation Contract, part-time Engagement Remote...Hourly payContract workPart timeFor contractorsRemote workFlexible hours- AuraOne, Inc. is seeking a Domain Contributor Evaluation Specialist for a remote contractor role in the United States. You will review AI outputs against workflows, policy, and stakeholder-fit using a structured rubric, and document the next steps for model training. Responsibilities...Remote jobFor contractors10 hours per week
- Community Access is a leading provider in New York City, delivering Integrated Mobile Treatment (IMT). The IMT Behavioral Health Specialist plays a key role within a multi-disciplinary team, engaging participants through assessment and therapeutic planning to support wellness...
- New York City Criminal Justice Agency (CJA) in Queens, NY seeks a Behavioral Health Specialist for intensive case management within the Queens Supervised Release program. You will provide community-based support to high-risk, high-need clients and work with a multidisciplinary...
- Allegheny Health Network seeks a Patient Access Coordinator for Behavioral Health in Wexford, PA. The role handles scheduling, preregistration, financial clearance, and insurance verification, ensuring accurate data and smooth patient flow. You will communicate clearly...
- Beth Israel Lahey Health, Inc. seeks a Bed Placement and Transfer Center Counselor to triage patient intake and transfers, ensuring timely... ...of transfers to ensure seamless patient care within our behavioral services network. #J-18808-Ljbffr Beth Israel Lahey Health
- Northeast Health Services, part of Transformations Care Network, seeks an Intake Call Center Representative to connect individuals with behavioral health services and support. You will handle screenings, guide clients to appropriate care, and document interactions with...Remote job
$8 - $65 per hour
Epidemiology Specialist - Freelance AI Trainer Project United States of America Are you a master of... ...world‑class education, accelerate public health insights, and support evidence-based... ...to our prompt engineering and evaluation metrics. A PhD degree in epidemiology...Remote jobHourly payContract workFor contractorsFreelanceImmediate start$21 - $35 per hour
Crossing Hurdles is seeking an Image Evaluation Expert (Visual Arts / Photography) on a remote contract basis. You will upload photos, compare AI-generated images with real photographs, and assess lighting, perspective, and textures to ensure visual accuracy. You will...Remote jobHourly payContract work- CASES in New York City seeks a Behavioral Health Specialist I to join a multidisciplinary team serving justice-involved individuals with serious mental illness. The role emphasizes integrated treatment planning, crisis intervention, medication management, and community...Flexible hours
- The Educational Alliance Inc. seeks a Clinical Assistant/Behavioral Health Screener in New York, NY. This full-time role combines front desk duties, initial behavioral health screenings, intake coordination, and insurance verification to ensure timely access to services...Full time
- Softechub is seeking an Image Evaluation Expert to remotely support our AI-driven image assessment projects in the United States. You will rate photos, provide detailed feedback, and help train next-generation AI systems with high-quality real-world input. The role emphasizes...Remote job
$25 - $35 per hour
...passionate about helping children and families thrive? Do you have experience working in mental/behavioral health? We are looking for dedicated Behavioral Health Specialists to provide individualized support services to youth and families in the foster care system who are...Part time- Center for Alternative Sentencing & Employment Services (CASES) in New York, NY seeks a Behavioral Health Specialist I to support justice-involved individuals with behavioral health needs. The role involves assessment, treatment planning, and linkages to medical care,...Full time
- Educational Alliance’s Center for Recovery and Wellness seeks a Clinical Assistant/Behavioral Health Screener to serve as the first point of contact for behavioral health services. The role combines front desk duties, initial screening, intake coordination, and insurance...
- Crossing Hurdles is seeking an Image Evaluation Expert for a remote contract role in the United States. You will upload photographs, compare real images with AI outputs, and assess lighting, perspective, and textures to ensure accuracy. You will provide clear, evidence-...Remote jobContract work10 hours per week
$63.65k
...Job Description Job Description Job Title: Behavioral Health Specialist I Job Summary: CASES is a New York City agency working at the forefront of the intersection between behavioral health and involvement in the criminal justice system. Our mission...Full timeWork at officeRemote workTrial periodMonday to FridayFlexible hoursShift workDay shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Behavioral Health Specialist - AI Evaluation. Be the first to apply!
- live chat specialist New York, NY
- patient safety specialist New York, NY
- credit card specialist New York, NY
- senior specialist New York, NY
- zoning specialist New York, NY
- charge entry specialist New York, NY
- health program specialist New York, NY
- youth specialist New York, NY
- microsoft specialist New York, NY
- transportation specialist New York, NY


