AI Safety Practitioner - Expert Evaluator
Mercor
We are seeking experienced AI Safety Practitioners to evaluate the safety, quality, and alignment of frontier AI models across complex, policy-sensitive, and ambiguous ("grey-area") topics. You will assess AI-generated responses, apply safety policies, and help improve model behavior through structured evaluations and feedback. Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality. Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains. Apply and refine evaluation rubrics for RLHF, SFT, and AI safety benchmarking. Identify unsafe outputs, hallucinations, reasoning failures, and policy violations. Provide structured feedback to improve model alignment and safety performance. Collaborate with AI researchers and safety teams on ongoing evaluation initiatives. Required Qualifications Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline. 5+ years of professional experience in AI Safety, Trust & Safety, journalism, public policy, scientific research, security, or a related field. Excellent written English, critical thinking, and analytical reasoning skills. Ability to consistently evaluate nuanced and policy-sensitive scenarios. Preferred Qualifications Experience with AI Safety, RLHF, SFT, Trust & Safety, or AI evaluation. Familiarity with safety policies, content moderation, or evaluation rubric development. Experience reviewing complex, high-risk, or ambiguous content. Why Join? Shape the safety and behaviour of frontier AI models used by millions worldwide. Work on challenging, real-world safety evaluations across nuanced and high-impact domains. Collaborate with leading AI researchers, engineers, and safety teams. #J-18808-Ljbffr Mercor
$60 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: AI Safety Practitioner Type: Contract Compensation:... ...Remote Role Responsibilities Evaluate AI-generated responses for safety, factual...SuggestedContract workSummer workRemote work- Obsidian is seeking experienced AI Safety Practitioners to evaluate safety, quality, and alignment of frontier AI models across complex policy-sensitive topics. You will assess AI-generated responses, apply safety policies, and help improve model behavior through structured...Suggested
- Obsidian is seeking expert Evaluators in FP&A / corporate finance to assess AI-generated work products for accuracy and quality. This role entails deep expertise to grade outputs and provide structured feedback. Candidates should have at least 5 years of relevant experience...SuggestedRemote jobHourly payWork at office
- Mercor, a San Francisco-based talent platform, is seeking a Government/public administration Evaluator for a remote contract. You will assess AI-generated artifacts against domain rubrics, identify factual and presentation errors, and provide structured feedback to improve...SuggestedRemote jobContract workWork at office
- Mercor is seeking experienced AI Safety Practitioners to evaluate frontier AI model safety, quality, and alignment across complex, policy-sensitive topics. You will assess AI-generated responses, apply safety policies, and help improve model behavior through structured...Suggested
- Mercor is seeking a remote AI red team specialist to probe and improve safety of conversational models. You will perform adversarial testing, annotate failures, and generate data to surface vulnerabilities and risks for customers. This role emphasizes structured methodologies...Remote job
- Mercor is seeking AI Safety Experts for English and Odia to remotely test and harden conversational AI models. You will simulate jailbreaks, annotate model failures, and classify vulnerabilities to help clients reduce risks and improve reliability. This contract role offers...Remote jobContract workFlexible hours
- Mercor is assembling a panel of radiological safety experts to red-team frontier AI models. The work centers on writing challenging, domain-specific... ...prompts across benign, dual-use, and adversarial levels, and evaluating model responses against a defined policy standard. You...Work at office
- Mercor connects elite creative and technical talent with leading AI research labs. We are seeking AI Safety Experts fluent in English and Indonesian for remote contract work. You will red-team conversational AI models, identify jailbreaks, prompt injections, misuse cases...Remote jobContract work
$16 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Adam D'Angelo , Larry Summers , and Jack Dorsey . Position AI Safety Experts — English & Odia Type Contract Compensation $16-$22/hour Location...Contract workSummer workRemote workFlexible hours- We are seeking experienced AI Safety Red Teamers to identify vulnerabilities in frontier AI systems through adversarial testing. You will design challenging prompts, uncover model weaknesses, and evaluate AI behavior across complex, high-risk, and ambiguous ("grey-area"...
$17 - $25 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Adam D'Angelo , Larry Summers , and Jack Dorsey . Position AI Safety Experts — English & Indonesian Type Contract Compensation $17-$25/hour...Contract workSummer workRemote work$20 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Bengali Type: Contract Compensation: $20...Contract workSummer workRemote work$20 - $22 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Angelo , Larry Summers , and Jack Dorsey . Position: AI Safety Experts — English & Punjabi Type: Contract Compensation: $20...Contract workSummer workRemote work$150 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...General Clinician (MD/DO) - HLS Expert Pool Type: Contract Compensation... ...model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and appropriate escalation...Contract workSummer workRemote work- ...assembling a panel of radiological safety experts to red-team frontier AI models. The goal is to test... ...benign, dual-use, and adversarial. Evaluate the responses that come back against... ...that line reliably is why we need practitioners rather than generalists. Background...
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Public-sector procurement / RFI response Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work$90 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...genuinely correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use of...Contract workSummer workLocal areaRemote work$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: AI Safety Practitioner Type: Contract Compensation: $60-$70/... ...Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual...Remote jobContract workSummer work$120 - $175 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Jack Dorsey . Position: CNC Machining Expert Type: Contract Compensation: $1... ...~ Strong production judgment to assess safety and feasibility on real machines. ~ Written...Contract workSummer workRemote work$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Dorsey . Position: Software Engineer Expert Type: Contract Compensation: $... ...Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...Contract workSummer workWork at officeRemote work- ...Evaluate AI model safety by designing adversarial test prompts and classifications. About Turing: Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced...Part timeFor contractors
$50 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...Position: English Language and Literature Expert Type: Contract Compensation: $50/hour Location: Remote Role Responsibilities Evaluate the quality and accuracy of LLM-generated...Contract workSummer workRemote work$70 - $84 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ..., and Jack Dorsey . Position: AI Safety Red Teamer Type: Contract Compensation... ...hallucinations, and policy failures. Evaluate model robustness across misinformation, cyber...Contract workSummer workRemote work$17 - $25 per hour
...Mercor, we believe the safest AI is the one that’s already been... ...for this project - human data experts who probe AI models with adversarial... ...customer AI systems Evaluation coverage expands: more scenarios... ...Mercor customers trust the safety of their AI because you’ve already...Hourly payContract workRemote work$80 - $150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Health Clinical Advisor (AI Safety Benchmark Project) Type:... ...and refine clinician-informed evaluation rubrics for AI chatbot responses... ...generated conversations and provide expert judgment to calibrate...Contract workTraineeshipSummer workRemote work- ...relevant experience in CFD or closely related engineering simulation work. Engagement: 15–20 hours/week Location: US, Canada, UK, or Europe Engagement Remote, project-based expert role Supporting AI model training and evaluation for a leading AI lab ....Hourly payFreelanceRemote work
$90 per hour
...small panel of nuclear and radiological professionals to audit how AI models respond to sensitive questions in the field. Each task... ...export control, national labs), health physicists and radiation safety officers, radiochemists, and former military or government nuclear...Contract work$50 - $70 per hour
...small panel of nuclear and radiological professionals to audit how AI models respond to sensitive questions in the field. Each task... ...export control, national labs), health physicists and radiation safety officers, radiochemists, and former military or government nuclear...Contract workTemporary workRemote work$100 per hour
...Description SME Careers is hiring chemists and chemical safety professionals to audit how AI models respond to sensitive questions in the field. Each... ...of diverse subjects. Optimize AI Performance: Evaluate and rank AI responses to enhance the model's accuracy, fluency...Permanent employmentFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Safety Practitioner - Expert Evaluator. Be the first to apply!
- practitioner San Francisco, CA
- holistic practitioner San Francisco, CA
- infection control practitioner San Francisco, CA
- subject matter expert San Francisco, CA
- technology expert San Francisco, CA
- fulfillment expert San Francisco, CA
- guest service support expert San Francisco, CA
- sql expert San Francisco, CA
- quality evaluator San Francisco, CA
- evaluator San Francisco, CA






