AI Safety Practitioner - Expert Evaluator
$60 - $70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Safety Practitioner
Type: Contract
Compensation: $60–$70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
- Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
- Apply and refine evaluation rubrics for RLHF , SFT , and AI safety benchmarking .
- Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
- Provide structured feedback to improve model alignment and safety performance.
- Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.
Qualifications
Must-Have
- Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline.
- 5+ years of professional experience in AI Safety , Trust & Safety, journalism, public policy, scientific research, security, or a related field.
- Excellent written English , critical thinking, and analytical reasoning skills.
- Ability to consistently evaluate nuanced and policy-sensitive scenarios.
Preferred
- Experience with AI Safety , RLHF , SFT , Trust & Safety, or AI evaluation.
- Familiarity with safety policies, content moderation, or evaluation rubric development.
- Experience reviewing complex, high-risk, or ambiguous content.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
$25 - $35 per hour
Review and improve AI answers written in Spanish so they read naturally and get the facts right. You rate responses, fix weak ones and write examples that show the model what good looks like. What you will do Rate Spanish AI responses for accuracy, fluency, tone and...SuggestedRemote jobHourly payWeekly payContract workPart timeFor contractorsFlexible hours- Synthires is seeking a Film & Video Craft Expert to remotely evaluate AI-generated videos across lighting, framing, color, and performance. This part-time role emphasizes concise feedback and strong written communication. Applicants should be fluent in English and have...SuggestedRemote jobPart time
- Vettedlance is seeking a meticulous Polish language reviewer to enhance AI answers. You will rate Polish responses for accuracy, fluency, and tone, rewrite weak answers, and ensure factual correctness. You will also fact-check claims, flag unsafe content, and craft prompts...SuggestedRemote jobHourly payPart timeFlexible hours
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location...SuggestedContract workSummer workWork at officeRemote work$30 - $50 per hour
OpenTrain AI, Inc. is seeking an AI Privacy, Safety and Evaluation Specialist for a six-week, part-time project based in the United States. You will test personal AI agents and review responses for privacy risks, with pay of $30 to $50 per hour. This remote contract offers...SuggestedRemote jobHourly payContract workPart timeRelocationFlexible hours- Appsierra Group is seeking an AI Session Annotator for a fully remote, independent contractor role. You will review recorded AI sessions, compare responses to visuals, and provide precise written feedback using a detailed rubric. Strong English, attention to detail, and...Remote jobWeekly payFor contractorsFlexible hours
- AuraOne, Inc. is seeking a remote Deception Behavior Risk Evaluator to stress-test AI systems with adversarial prompts. You will design jailbreak... ...applicable policy clause to enable rapid patching by the safety team. This contractor role requires strong written communication...Remote jobFor contractors
$65 - $70 per hour
...Biology Expert (PhD) — AI Safety is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack... ...violated so the safety team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before they ship to...For contractorsRemote work10 hours per week- Mercor is hiring for a part-time Danish-English evaluative role to help make advanced AI models safer. You will craft expert-level prompts in Danish and apply guidelines to classify prompts and conversations, documenting your judgments with clear reasoning. No prior AI...Part timeImmediate start
- ...contractor Remote Contract Energetic Materials Explosives Engineering AI Safety Hazmat Safety What this pays Rates are set by the client... ...a figure we cannot stand behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US, Canada, UK, Europe,...Hourly payContract workFor contractorsRemote work
- ...seeking bilingual Spanish-English speakers to help improve AI safety by evaluating how models handle sensitive topics in Spanish. No prior AI/... ...experience required; training will be provided. You will write expert prompts in Spanish, apply structured guidelines to classify...Part timeImmediate start
$35 - $65 per hour
Review AI medication answers, write expert pharmacy responses and identify safety issues. Work is remote and part-time, and you start paid tasks after a short recorded interview. What you will do Rate AI medication answers Write expert responses Identify interactions and...Hourly payWeekly payContract workPart timeFor contractorsRemote workFlexible hours$62.5 per hour
HVAC Expert for AI Diagnostics and System Design Build HVAC diagnostic and system-design scenarios... ...and rate AI responses for accuracy and safety. This remote contractor role requires... ...fixed schedule. Pay is up to $30 per hour. Evaluate and rank AI-generated responses in...Hourly payContract workPart timeFor contractorsRemote workFlexible hours$50 - $54 per hour
Bilingual Polish STEM Expert (PhD) — AI Safety is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers... ...so the safety team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before they ship to customers...For contractorsRemote work10 hours per week- AuraOne is seeking a Bilingual Polish STEM Expert (PhD) for a remote AI Safety red-team role. You will design adversarial prompts, document failures with precise reproduction steps, and rate defenses across turn-based conversations to harden AI models before release. Remote...For contractorsRemote work
$70 - $84 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Larry Summers, and Jack Dorsey. Position: AI Safety Red TeamerType:ContractCompensation:$70-$... ...behaviors, hallucinations, and policy failures. Evaluate model robustness across misinformation,...Summer workRemote work$40 per hour
...Type: Independent contractor · Experience: 3+ years experience About the Role Pasiflora AI has immediate openings for Chemical AI Safety Annotators & Subject Matter Experts to evaluate AI-generated responses in chemistry and chemical-safety scenarios. We're looking for...For contractorsImmediate startRemote work$43 - $47 per hour
...Spanish) to help make advanced AI models safer. You'll apply... ...fluency and cultural judgment to evaluate and strengthen how these models... .... What you'll do Write expert-level prompts in Spanish across... ...content. Background in trust and safety, content moderation, policy evaluation...Part timeImmediate start$40 per hour
Pasiflora AI seeks experienced chemistry professionals for remote Independent Contractor roles as Chemical AI Safety Annotators & Subject Matter Experts. You will evaluate AI responses in chemistry and safety, classify content, write expert references, and help calibrate...Remote jobFor contractors$40 - $44 per hour
Bilingual Portuguese Generalist Expert — AI Safety is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers... ...violated so the safety team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before they ship to...For contractorsRemote work10 hours per week- Mercor is seeking experienced Nurse Practitioners to evaluate AI-generated clinical content and workflows, applying real-world expertise in diagnosing and managing patient care. This fully remote role focuses on evaluating AI-driven clinical assessments, treatment plans...Remote jobWeekly payImmediate start
$37 per unit
...technical talent with leading AI research labs. Headquartered in... ...Dorsey . Position: AI Safety Specialist Type: Contract... ...Remote Role Responsibilities Evaluate the accuracy and depth of AI-... ...with subject matter experts to ensure consistency, relevance...Contract workSummer workRemote work$100 - $120 per unit
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Baseball Fan – Live MLB Game AI Evaluator (US) Type: Contract Compensation:... ..., schedules, broadcasts, highlights, and expert-style analysis. Run the same question...Contract workWork experience placementSummer workRemote work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work- Joinkeywork is seeking a chemistry reviewer to validate AI-generated chemistry content. You will write problems, review peers, and assess mechanisms, stoichiometry, and procedures for safety. Strong written English and ability to justify conclusions are essential. You...Remote work
$85 per hour
Process Safety Engineer AI Training Evaluator Review AI-generated content on EHS, process safety, and energy compliance using your professional experience. This remote, flexible project pays up to $85 per hour. Remote Hourly $85/hr $85/hr Compensation Contract Engagement...Hourly payContract workRemote workFlexible hours$68 per hour
Registered Nurse AI Evaluation Expert (English) Use your nursing judgment to review AI patient education, triage guidance, clinical documentation... ...train and assess AI systems. Your work focuses on accuracy, safety, clarity, and alignment with nursing practice. Review AI-...Hourly payContract workPart timeFor contractorsRemote workFlexible hours- ...the surgical setting. You will assess and implement anesthesia plans, monitor patients, and collaborate with physicians to ensure safety and quality. The role emphasizes compliance with HIPAA, regulatory standards, and continuous professional development. Prior CRNA experience...Part time
- Invisible Technologies Inc. seeks an AI QA trainer to harden model reasoning and reliability... ...failure modes to raise standards for safety, bias checks, and prompt robustness. As a... ...with researchers and engineers to evaluate LLM outputs, identify edge cases, and propose...Remote jobFor contractorsFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Safety Practitioner - Expert Evaluator. Be the first to apply!
- holistic practitioner New York, NY
- practitioner New York, NY
- assistant practitioner New York, NY
- stretch practitioner New York, NY
- infection control practitioner New York, NY
- technology expert New York, NY
- subject matter expert New York, NY
- guest service support expert New York, NY
- sql expert New York, NY
- fulfillment expert New York, NY


