Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Behavioral Health Expert, AI Safety and Model Evaluation

$45 - $70 per hour

AuraOne, Inc.

Behavioral Health Expert, AI Safety and Model Evaluation is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team can patch the gap. Why this role matters Adversarial evaluation is how AuraOne hardens AI models before they ship to customers. Reviewers think like attackers and write up failures with enough rigor that the modeling team can reproduce, fix, and regress-test them. Responsibilities Design adversarial prompts that probe known weakness classes (jailbreak, policy bypass, prompt injection) for Behavioral Health Expert, AI Safety and Model Evaluation assignments. Document every successful attack with reproduction steps and the policy clause it violated. Score model defenses across single-turn and multi-turn conversations. Triage emerging attack vectors and route them to the safety team with severity ratings. Maintain a personal library of attack patterns and propose new red-team rubrics. Calibrate against the broader red-team cohort to keep coverage and severity consistent. Qualifications Demonstrated experience red-teaming AI systems, security research, or adversarial ML work for Behavioral Health Expert, AI Safety and Model Evaluation work. Strong written communication — your reports become the patch ticket. Comfort working in policy-grey areas with clear documentation of what was attempted and why. Familiarity with prompt-injection, jailbreak, and policy-bypass taxonomies. Reliable async availability for at least 10 hours per week. Example tasks Construct a 5-turn adversarial conversation that bypasses a specific policy clause and write up the patch ticket. Score a model's defenses against a known jailbreak pattern across 20 variants. Propose a new red-team rubric category after spotting an emerging attack vector. Reproduce a failure another reviewer reported and confirm the severity tag. Nice to have Background in offensive security, AppSec, or trust & safety operations. Experience publishing or reproducing public adversarial-ML research. Multilingual fluency for cross-language attack testing. Skills Adversarial prompting Red-team analysis Policy taxonomy Failure documentation Behavioral Health AI Safety and Model Evaluation evaluation Work model Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US. Compensation $45–$70 / hr Demonstrated experience red-teaming AI systems, security research, or adversarial ML work for Behavioral Health Expert, AI Safety and Model Evaluation work. Strong written communication — your reports become the patch ticket. Comfort working in policy-grey areas with clear documentation of what was attempted and why. Familiarity with prompt-injection, jailbreak, and policy-bypass taxonomies. Reliable async availability for at least 10 hours per week. #J-18808-Ljbffr

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Behavioral Health Expert, AI Safety and Model Evaluation in New York, NY vacancy
  • 1. Overview A leading AI lab is seeking behavioral health experts to help evaluate and improve how its AI models handle sensitive, everyday conversations. People increasingly turn...  ...feedback. Nice to have: a background in AI safety, applied ethics, trust and safety or... 
    Suggested
    Contract work
    Part time
    Weekday work

    Mercor

    New York, NY
    2 days ago
  • $150 per hour

    Mercor is seeking a General Physician for a remote, short-term contract. You will evaluate AI model outputs for clinical validity, provide structured feedback, and flag safety issues while collaborating with AI researchers. The assignment spans 6 weeks at ~20+ hours per... 
    Suggested
    Remote job
    Hourly pay
    Temporary work

    mercor

    New York, NY
    4 days ago
  • Cincinnatus LLC is seeking behavioral health experts to help evaluate how AI models respond in everyday conversations about relationships, wellbeing and beliefs. This part-time role focuses on neutrality, safety and balanced guidance, with 20-40 hours per week and a W-... 
    Suggested
    Part time

    Annotation Academy

    New York, NY
    1 day ago
  • Cincinnatus LLC is seeking behavioral health experts to evaluate how AI models respond in sensitive conversations, including relationships, wellbeing, and personal beliefs. This is a part‑time W‑2 role with 20-40 hours per week, placed at a leading AI lab as part of the... 
    Suggested
    Part time

    Mercor

    New York, NY
    4 days ago
  •  ...pace of frontier model development. They...  ...building model evals and safety from the ground up...  ...defining the evaluation frameworks, building...  ...evaluation and AI safety, including...  ...topics like model behavior, evaluation design...  ...through stock options.Health & wellness:... 
    Suggested
    Work at office
    Visa sponsorship

    Reflection AI

    New York, NY
    4 days ago
  • $110 per hour

     ...background and interests. Role Overview Qualified physicians may contribute to AI research projects by bringing real-world clinical expertise to model development and evaluation. Key Responsibilities Train and evaluate AI models in medicine. Create tasks and... 
    Hourly pay
    Contract work
    Remote work

    SaidGig

    New York, NY
    3 days ago
  •  ...creative and technical talent with leading AI research labs. The company is...  ...Summers, and Jack Dorsey. They are hiring a AI Safety Practitioner for a contract role with remote work options. The role focuses on evaluating AI-generated responses for safety, accuracy... 
    Remote job
    Contract work
    Summer work

    Mercor Inc

    New York, NY
    4 days ago
  • A leading AI research firm in New York, NY, is looking for a specialist to oversee...  ...the red-teaming and adversarial evaluation pipeline for their models. The ideal candidate should possess a...  ...and have a deep understanding of LLM safety and adversarial techniques. A strong... 

    Reflection AI

    New York, NY
    4 days ago
  • $50 - $70 per hour

     ...radiological professionals to audit how AI models respond to sensitive questions in the field...  ...: the kinds of things reactor engineers, health physicists, regulators, students and...  ...national labs), health physicists and radiation safety officers, radiochemists, and former... 
    Remote job
    Temporary work

    Braintrust

    New York, NY
    5 days ago
  •  ...clinical expertise to the development and evaluation of next-generation AI systems. This remote, contract...  ...realistic clinical scenarios, develop expert reference responses, and evaluate AI...  ...clinical accuracy, ethical practice, safety, and risk management. #J-18808-Ljbffr... 
    Remote job
    Contract work

    Appsierra Group

    New York, NY
    3 days ago
  • Mercor is seeking experienced MDs or DOs to evaluate health AI models remotely in the United States. You will review outputs across diagnoses, clinical...  ...provide structured feedback to ensure medical accuracy and safety. The role requires an active US license, completion of... 
    Remote job
    Part time
    For contractors

    Benture

    New York, NY
    2 days ago
  •  ...Mercor is partnering with a leading AI research lab to train frontier models on clinical and biomedical reasoning. We’re hiring Healthcare Experts to evaluate model outputs and ensure medical...  ...provide feedback to improve model behavior. Requirements include an MD/DO/PharmD... 

    Mercor Inc

    New York, NY
    4 days ago
  •  ...Cincinnatus LLC is hiring for a Marketing SME to support GenAI model evaluation and brand/growth tasks. The role centers on applying rigorous marketing judgment to AI training data and guiding cross-functional teams to improve model outputs. Ideal candidates have extensive... 

    Mercor Inc

    New York, NY
    4 days ago
  • $48 - $62 per hour

     ...strengthen conversational AI systems by probing them...  ...producing practical safety data. This remote role...  ...Test conversational AI models and agents for...  ...misinformation, and harmful behaviors Annotate failures, classify...  ...may miss. Expand evaluation coverage across more... 
    Hourly pay
    Remote work

    SaidGig

    New York, NY
    3 days ago
  • $50 per hour

     ...Help evaluate and improve advanced AI language systems by applying expert English-language judgment to model outputs and training data. This fully remote role focuses on assessing how well large language models understand, generate, and use English across a range of real... 
    Hourly pay
    Remote work
    Flexible hours

    SaidGig

    New York, NY
    2 days ago
  • $50 - $70 per hour

     ...SuperAnnotate AI, Inc is assembling a panel of nuclear and radiological professionals for AI model auditing tasks. This short-term project seeks experts in reactor physics, health physics, safeguards, and related fields to classify and critique model responses on sensitive... 
    Hourly pay
    Temporary work

    Brain Trust Inc

    New York, NY
    4 days ago
  •  ...Mercor is seeking AI Safety Experts to join our remote contract team. This role focuses on evaluating and improving the safety of AI systems...  ...will red-team conversational models, annotate failures, and...  ...help partners reduce risky behavior and system vulnerabilities.... 
    Contract work
    Remote work
    Flexible hours

    Mercor Inc

    New York, NY
    3 days ago
  • $65 per hour

     ...Prolific is seeking Registered Nurses to help train and evaluate AI models. If you have clinical experience, we’ll test your skills and, if successful, invite you to join as a Domain Expert participant, where you’ll be paid to contribute. Researchers may pay up to $65/... 
    Remote work
    Work from home
    Flexible hours

    Prolific

    New York, NY
    3 days ago
  • $60 - $90 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Machine Learning Engineer — Model Evaluation & Experimentation Type:...  ...reward functions and training behavior. Evaluate models to identify...  ...Collaborate with researchers and experts to maintain task consistency... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    26 days ago
  • $48 - $62 per hour

    Probe conversational AI systems for weaknesses before...  ..., create high-quality safety data, and produce...  ...misinformation, and harmful behaviors. Participation in...  ...team conversational AI models and agents through jailbreaks...  ...risks. Help expand evaluation coverage by testing more... 
    Hourly pay
    Remote work

    SaidGig

    New York, NY
    3 days ago
  • $60 per hour

    Prolific is seeking Chemistry Experts and Chemical Engineers to join our Expert Network to train and evaluate AI models using real-world chemical expertise. If successful, you’ll be invited to join Prolific as a participant, with pay up to $60 per hour and flexible hours... 
    Remote job
    Hourly pay
    Flexible hours

    Prolific

    New York, NY
    2 days ago
  • $45 - $90 per hour

     ...bioinformatics workflows used in AI benchmarking and model development. The work covers...  ...recommended practices. Design, evaluate, and critique biology, bioinformatics...  ...artificial intelligence. Experts review examples, test model behavior, and assess whether outputs and... 
    Hourly pay
    Contract work
    Part time

    OpenTrain AI, Inc.

    New York, NY
    3 days ago
  •  ...we believe the safest AI is the one that’s...  ...project - human data experts who probe AI models with adversarial inputs...  ...misinformation, or harmful behaviors. All work is text-...  ...customer AI systems Evaluation coverage expands:...  ...customers trust the safety of their AI because you... 
    Remote work

    Mercor

    New York, NY
    4 days ago
  • $100 per hour

     ...to help improve next-generation AI systems. This remote contract role focuses on evaluating psychological content, developing...  ...appropriate materials, and providing expert feedback that supports accurate, ethical, and context-sensitive model training. Prior AI experience is... 
    Hourly pay
    Contract work
    Remote work

    SaidGig

    New York, NY
    2 days ago
  • $60 - $70 per hour

     ...technical talent with leading AI research labs. Headquartered in...  ...Dorsey . Position: AI Safety Practitioner Type: Contract...  ...Role Responsibilities Evaluate AI-generated responses for safety...  ...feedback to improve model alignment and safety performance... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    19 days ago
  • $60 - $80 per hour

     ...advanced large language models. In this role, you will...  ...campaign judgment to AI training data, partnering...  .... Develop and improve evaluation guidelines and scoring...  ...with other subject-matter experts to maintain consistent,...  ..., reproductive health decisions, or related medical... 
    Hourly pay
    Weekday work

    SaidGig

    New York, NY
    2 days ago
  • $15 - $35 per hour

     ...Help Shape the Future of AI in Healthcare What if...  ...healthcare information, evaluates medical scenarios, and...  ...Artificial Intelligence (AI) models. We're seeking...  ...Preventive care, wellness, and health screenings....  ...and potential patient safety concerns in AI-generated... 
    Hourly pay
    For contractors
    Traineeship
    Freelance
    Internship
    Remote work
    Weekday work

    Volga Partners

    New York, NY
    4 days ago
  •  ...internet safer. You don’t need to be a tech expert; your human insight will guide how AI-generated content is evaluated for safety, usefulness, and responsibility. You’ll review...  ...where advice may be unsafe, applying mental health knowledge to ensure sensitive topics are... 
    Freelance
    Remote work

    TELUS Digital AI Solution

    New York, NY
    2 days ago
  •  ...TELUS Digital AI Community seeks psychology experts to review AI content for safety and responsibility. Use your psychology...  ...to help AI understand human behavior and mental health considerations, acting as a...  ...empathy and ethics to guide model development across global... 
    Remote work

    Jobs for Humanity

    New York, NY
    5 hours ago
  • $174k - $252k

     ...both human-powered and Large Language Model (LLM)-powered automated evaluation systems to assess model performance....  ...aspects like grounding, coherence, safety, and helpfulness.Utilize platforms...  ...projects.Experience integrating generative AI tools or Large Language Model (LLM)... 

    Google

    New York, NY
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Behavioral Health Expert, AI Safety and Model Evaluation. Be the first to apply!