Behavioral Health Expert, AI Safety and Model Evaluation
$45 - $70 per hourAuraOne, Inc.
Behavioral Health Expert, AI Safety and Model Evaluation is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team can patch the gap. Why this role matters Adversarial evaluation is how AuraOne hardens AI models before they ship to customers. Reviewers think like attackers and write up failures with enough rigor that the modeling team can reproduce, fix, and regress-test them. Responsibilities Design adversarial prompts that probe known weakness classes (jailbreak, policy bypass, prompt injection) for Behavioral Health Expert, AI Safety and Model Evaluation assignments. Document every successful attack with reproduction steps and the policy clause it violated. Score model defenses across single-turn and multi-turn conversations. Triage emerging attack vectors and route them to the safety team with severity ratings. Maintain a personal library of attack patterns and propose new red-team rubrics. Calibrate against the broader red-team cohort to keep coverage and severity consistent. Qualifications Demonstrated experience red-teaming AI systems, security research, or adversarial ML work for Behavioral Health Expert, AI Safety and Model Evaluation work. Strong written communication — your reports become the patch ticket. Comfort working in policy-grey areas with clear documentation of what was attempted and why. Familiarity with prompt-injection, jailbreak, and policy-bypass taxonomies. Reliable async availability for at least 10 hours per week. Example tasks Construct a 5-turn adversarial conversation that bypasses a specific policy clause and write up the patch ticket. Score a model's defenses against a known jailbreak pattern across 20 variants. Propose a new red-team rubric category after spotting an emerging attack vector. Reproduce a failure another reviewer reported and confirm the severity tag. Nice to have Background in offensive security, AppSec, or trust & safety operations. Experience publishing or reproducing public adversarial-ML research. Multilingual fluency for cross-language attack testing. Skills Adversarial prompting Red-team analysis Policy taxonomy Failure documentation Behavioral Health AI Safety and Model Evaluation evaluation Work model Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US. Compensation $45–$70 / hr Demonstrated experience red-teaming AI systems, security research, or adversarial ML work for Behavioral Health Expert, AI Safety and Model Evaluation work. Strong written communication — your reports become the patch ticket. Comfort working in policy-grey areas with clear documentation of what was attempted and why. Familiarity with prompt-injection, jailbreak, and policy-bypass taxonomies. Reliable async availability for at least 10 hours per week. #J-18808-Ljbffr
- 1. Overview A leading AI lab is seeking behavioral health experts to help evaluate and improve how its AI models handle sensitive, everyday conversations. People increasingly turn... ...feedback. Nice to have: a background in AI safety, applied ethics, trust and safety or...SuggestedContract workPart timeWeekday work
$150 per hour
Mercor is seeking a General Physician for a remote, short-term contract. You will evaluate AI model outputs for clinical validity, provide structured feedback, and flag safety issues while collaborating with AI researchers. The assignment spans 6 weeks at ~20+ hours per...SuggestedRemote jobHourly payTemporary work- Cincinnatus LLC is seeking behavioral health experts to help evaluate how AI models respond in everyday conversations about relationships, wellbeing and beliefs. This part-time role focuses on neutrality, safety and balanced guidance, with 20-40 hours per week and a W-...SuggestedPart time
- Cincinnatus LLC is seeking behavioral health experts to evaluate how AI models respond in sensitive conversations, including relationships, wellbeing, and personal beliefs. This is a part‑time W‑2 role with 20-40 hours per week, placed at a leading AI lab as part of the...SuggestedPart time
- ...pace of frontier model development. They... ...building model evals and safety from the ground up... ...defining the evaluation frameworks, building... ...evaluation and AI safety, including... ...topics like model behavior, evaluation design... ...through stock options.Health & wellness:...SuggestedWork at officeVisa sponsorship
$110 per hour
...background and interests. Role Overview Qualified physicians may contribute to AI research projects by bringing real-world clinical expertise to model development and evaluation. Key Responsibilities Train and evaluate AI models in medicine. Create tasks and...Hourly payContract workRemote work- ...creative and technical talent with leading AI research labs. The company is... ...Summers, and Jack Dorsey. They are hiring a AI Safety Practitioner for a contract role with remote work options. The role focuses on evaluating AI-generated responses for safety, accuracy...Remote jobContract workSummer work
- A leading AI research firm in New York, NY, is looking for a specialist to oversee... ...the red-teaming and adversarial evaluation pipeline for their models. The ideal candidate should possess a... ...and have a deep understanding of LLM safety and adversarial techniques. A strong...
$50 - $70 per hour
...radiological professionals to audit how AI models respond to sensitive questions in the field... ...: the kinds of things reactor engineers, health physicists, regulators, students and... ...national labs), health physicists and radiation safety officers, radiochemists, and former...Remote jobTemporary work- ...clinical expertise to the development and evaluation of next-generation AI systems. This remote, contract... ...realistic clinical scenarios, develop expert reference responses, and evaluate AI... ...clinical accuracy, ethical practice, safety, and risk management. #J-18808-Ljbffr...Remote jobContract work
- Mercor is seeking experienced MDs or DOs to evaluate health AI models remotely in the United States. You will review outputs across diagnoses, clinical... ...provide structured feedback to ensure medical accuracy and safety. The role requires an active US license, completion of...Remote jobPart timeFor contractors
- ...Mercor is partnering with a leading AI research lab to train frontier models on clinical and biomedical reasoning. We’re hiring Healthcare Experts to evaluate model outputs and ensure medical... ...provide feedback to improve model behavior. Requirements include an MD/DO/PharmD...
- ...Cincinnatus LLC is hiring for a Marketing SME to support GenAI model evaluation and brand/growth tasks. The role centers on applying rigorous marketing judgment to AI training data and guiding cross-functional teams to improve model outputs. Ideal candidates have extensive...
$48 - $62 per hour
...strengthen conversational AI systems by probing them... ...producing practical safety data. This remote role... ...Test conversational AI models and agents for... ...misinformation, and harmful behaviors Annotate failures, classify... ...may miss. Expand evaluation coverage across more...Hourly payRemote work$50 per hour
...Help evaluate and improve advanced AI language systems by applying expert English-language judgment to model outputs and training data. This fully remote role focuses on assessing how well large language models understand, generate, and use English across a range of real...Hourly payRemote workFlexible hours$50 - $70 per hour
...SuperAnnotate AI, Inc is assembling a panel of nuclear and radiological professionals for AI model auditing tasks. This short-term project seeks experts in reactor physics, health physics, safeguards, and related fields to classify and critique model responses on sensitive...Hourly payTemporary work- ...Mercor is seeking AI Safety Experts to join our remote contract team. This role focuses on evaluating and improving the safety of AI systems... ...will red-team conversational models, annotate failures, and... ...help partners reduce risky behavior and system vulnerabilities....Contract workRemote workFlexible hours
$65 per hour
...Prolific is seeking Registered Nurses to help train and evaluate AI models. If you have clinical experience, we’ll test your skills and, if successful, invite you to join as a Domain Expert participant, where you’ll be paid to contribute. Researchers may pay up to $65/...Remote workWork from homeFlexible hours$60 - $90 per hour
...technical talent with leading AI research labs. Headquartered... ...Machine Learning Engineer — Model Evaluation & Experimentation Type:... ...reward functions and training behavior. Evaluate models to identify... ...Collaborate with researchers and experts to maintain task consistency...Contract workSummer workRemote work$48 - $62 per hour
Probe conversational AI systems for weaknesses before... ..., create high-quality safety data, and produce... ...misinformation, and harmful behaviors. Participation in... ...team conversational AI models and agents through jailbreaks... ...risks. Help expand evaluation coverage by testing more...Hourly payRemote work$60 per hour
Prolific is seeking Chemistry Experts and Chemical Engineers to join our Expert Network to train and evaluate AI models using real-world chemical expertise. If successful, you’ll be invited to join Prolific as a participant, with pay up to $60 per hour and flexible hours...Remote jobHourly payFlexible hours$45 - $90 per hour
...bioinformatics workflows used in AI benchmarking and model development. The work covers... ...recommended practices. Design, evaluate, and critique biology, bioinformatics... ...artificial intelligence. Experts review examples, test model behavior, and assess whether outputs and...Hourly payContract workPart time- ...we believe the safest AI is the one that’s... ...project - human data experts who probe AI models with adversarial inputs... ...misinformation, or harmful behaviors. All work is text-... ...customer AI systems Evaluation coverage expands:... ...customers trust the safety of their AI because you...Remote work
$100 per hour
...to help improve next-generation AI systems. This remote contract role focuses on evaluating psychological content, developing... ...appropriate materials, and providing expert feedback that supports accurate, ethical, and context-sensitive model training. Prior AI experience is...Hourly payContract workRemote work$60 - $70 per hour
...technical talent with leading AI research labs. Headquartered in... ...Dorsey . Position: AI Safety Practitioner Type: Contract... ...Role Responsibilities Evaluate AI-generated responses for safety... ...feedback to improve model alignment and safety performance...Contract workSummer workRemote work$60 - $80 per hour
...advanced large language models. In this role, you will... ...campaign judgment to AI training data, partnering... .... Develop and improve evaluation guidelines and scoring... ...with other subject-matter experts to maintain consistent,... ..., reproductive health decisions, or related medical...Hourly payWeekday work$15 - $35 per hour
...Help Shape the Future of AI in Healthcare What if... ...healthcare information, evaluates medical scenarios, and... ...Artificial Intelligence (AI) models. We're seeking... ...Preventive care, wellness, and health screenings.... ...and potential patient safety concerns in AI-generated...Hourly payFor contractorsTraineeshipFreelanceInternshipRemote workWeekday work- ...internet safer. You don’t need to be a tech expert; your human insight will guide how AI-generated content is evaluated for safety, usefulness, and responsibility. You’ll review... ...where advice may be unsafe, applying mental health knowledge to ensure sensitive topics are...FreelanceRemote work
- ...TELUS Digital AI Community seeks psychology experts to review AI content for safety and responsibility. Use your psychology... ...to help AI understand human behavior and mental health considerations, acting as a... ...empathy and ethics to guide model development across global...Remote work
$174k - $252k
...both human-powered and Large Language Model (LLM)-powered automated evaluation systems to assess model performance.... ...aspects like grounding, coherence, safety, and helpfulness.Utilize platforms... ...projects.Experience integrating generative AI tools or Large Language Model (LLM)...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Behavioral Health Expert, AI Safety and Model Evaluation. Be the first to apply!
- subject matter expert New York, NY
- technology expert New York, NY
- fulfillment expert New York, NY
- guest service support expert New York, NY
- sql expert New York, NY
- subject matter expert work from home New York, NY
- health and safety New York, NY
- ehs intern New York, NY
- vice president of environmental health and safety New York, NY
- hse intern New York, NY



