Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Safety Practitioner - Expert Evaluator

$60 - $70 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: AI Safety Practitioner
Type: Contract
Compensation: $60–$70/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
  • Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
  • Apply and refine evaluation rubrics for RLHF , SFT , and AI safety benchmarking .
  • Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
  • Provide structured feedback to improve model alignment and safety performance.
  • Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.

Qualifications

Must-Have

  • Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline.
  • 5+ years of professional experience in AI Safety , Trust & Safety, journalism, public policy, scientific research, security, or a related field.
  • Excellent written English , critical thinking, and analytical reasoning skills.
  • Ability to consistently evaluate nuanced and policy-sensitive scenarios.

Preferred

  • Experience with AI Safety , RLHF , SFT , Trust & Safety, or AI evaluation.
  • Familiarity with safety policies, content moderation, or evaluation rubric development.
  • Experience reviewing complex, high-risk, or ambiguous content.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the AI Safety Practitioner - Expert Evaluator in New York, NY vacancy
  • $25 - $35 per hour

    Review and improve AI answers written in Spanish so they read naturally and get the facts right. You rate responses, fix weak ones and write examples that show the model what good looks like. What you will do Rate Spanish AI responses for accuracy, fluency, tone and... 
    Suggested
    Remote job
    Hourly pay
    Weekly pay
    Contract work
    Part time
    For contractors
    Flexible hours

    Vettedlance

    New York, NY
    3 days ago
  • Synthires is seeking a Film & Video Craft Expert to remotely evaluate AI-generated videos across lighting, framing, color, and performance. This part-time role emphasizes concise feedback and strong written communication. Applicants should be fluent in English and have... 
    Suggested
    Remote job
    Part time

    Synthires

    New York, NY
    5 days ago
  • Vettedlance is seeking a meticulous Polish language reviewer to enhance AI answers. You will rate Polish responses for accuracy, fluency, and tone, rewrite weak answers, and ensure factual correctness. You will also fact-check claims, flag unsafe content, and craft prompts... 
    Suggested
    Remote job
    Hourly pay
    Part time
    Flexible hours

    Vettedlance

    New York, NY
    2 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    9 days ago
  • $30 - $50 per hour

    OpenTrain AI, Inc. is seeking an AI Privacy, Safety and Evaluation Specialist for a six-week, part-time project based in the United States. You will test personal AI agents and review responses for privacy risks, with pay of $30 to $50 per hour. This remote contract offers... 
    Suggested
    Remote job
    Hourly pay
    Contract work
    Part time
    Relocation
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • Appsierra Group is seeking an AI Session Annotator for a fully remote, independent contractor role. You will review recorded AI sessions, compare responses to visuals, and provide precise written feedback using a detailed rubric. Strong English, attention to detail, and... 
    Remote job
    Weekly pay
    For contractors
    Flexible hours

    Recruitment Room

    New York, NY
    4 days ago
  • AuraOne, Inc. is seeking a remote Deception Behavior Risk Evaluator to stress-test AI systems with adversarial prompts. You will design jailbreak...  ...applicable policy clause to enable rapid patching by the safety team. This contractor role requires strong written communication... 
    Remote job
    For contractors

    AuraOne Human Data

    New York, NY
    4 days ago
  • $65 - $70 per hour

     ...Biology Expert (PhD) — AI Safety is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack...  ...violated so the safety team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before they ship to... 
    For contractors
    Remote work
    10 hours per week

    AuraOne

    New York, NY
    2 days ago
  • Mercor is hiring for a part-time Danish-English evaluative role to help make advanced AI models safer. You will craft expert-level prompts in Danish and apply guidelines to classify prompts and conversations, documenting your judgments with clear reasoning. No prior AI... 
    Part time
    Immediate start

    Mercor

    New York, NY
    6 days ago
  •  ...contractor Remote Contract Energetic Materials Explosives Engineering AI Safety Hazmat Safety What this pays Rates are set by the client...  ...a figure we cannot stand behind. About the Role AI Safety Evaluation Project | Remote | Contract Locations: US, Canada, UK, Europe,... 
    Hourly pay
    Contract work
    For contractors
    Remote work

    Pasiflora AI LLC

    New York, NY
    2 days ago
  •  ...seeking bilingual Spanish-English speakers to help improve AI safety by evaluating how models handle sensitive topics in Spanish. No prior AI/...  ...experience required; training will be provided. You will write expert prompts in Spanish, apply structured guidelines to classify... 
    Part time
    Immediate start

    Mercor

    New York, NY
    6 days ago
  • $35 - $65 per hour

    Review AI medication answers, write expert pharmacy responses and identify safety issues. Work is remote and part-time, and you start paid tasks after a short recorded interview. What you will do Rate AI medication answers Write expert responses Identify interactions and... 
    Hourly pay
    Weekly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    Vettedlance

    New York, NY
    3 days ago
  • $62.5 per hour

    HVAC Expert for AI Diagnostics and System Design Build HVAC diagnostic and system-design scenarios...  ...and rate AI responses for accuracy and safety. This remote contractor role requires...  ...fixed schedule. Pay is up to $30 per hour. Evaluate and rank AI-generated responses in... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    2 days ago
  • $50 - $54 per hour

    Bilingual Polish STEM Expert (PhD) — AI Safety is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers...  ...so the safety team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before they ship to customers... 
    For contractors
    Remote work
    10 hours per week

    Auraone

    New York, NY
    1 day ago
  • AuraOne is seeking a Bilingual Polish STEM Expert (PhD) for a remote AI Safety red-team role. You will design adversarial prompts, document failures with precise reproduction steps, and rate defenses across turn-based conversations to harden AI models before release. Remote... 
    For contractors
    Remote work

    Auraone

    New York, NY
    2 days ago
  • $70 - $84 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Larry Summers, and Jack Dorsey. Position: AI Safety Red TeamerType:ContractCompensation:$70-$...  ...behaviors, hallucinations, and policy failures. Evaluate model robustness across misinformation,... 
    Summer work
    Remote work

    mercor

    New York, NY
    4 days ago
  • $40 per hour

     ...Type: Independent contractor · Experience: 3+ years experience About the Role Pasiflora AI has immediate openings for Chemical AI Safety Annotators & Subject Matter Experts to evaluate AI-generated responses in chemistry and chemical-safety scenarios. We're looking for... 
    For contractors
    Immediate start
    Remote work

    Pasiflora AI LLC

    New York, NY
    5 days ago
  • $43 - $47 per hour

     ...Spanish) to help make advanced AI models safer. You'll apply...  ...fluency and cultural judgment to evaluate and strengthen how these models...  .... What you'll do Write expert-level prompts in Spanish across...  ...content. Background in trust and safety, content moderation, policy evaluation... 
    Part time
    Immediate start

    Mercor

    New York, NY
    6 days ago
  • $40 per hour

    Pasiflora AI seeks experienced chemistry professionals for remote Independent Contractor roles as Chemical AI Safety Annotators & Subject Matter Experts. You will evaluate AI responses in chemistry and safety, classify content, write expert references, and help calibrate... 
    Remote job
    For contractors

    Pasiflora AI LLC

    New York, NY
    5 days ago
  • $40 - $44 per hour

    Bilingual Portuguese Generalist Expert — AI Safety is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers...  ...violated so the safety team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before they ship to... 
    For contractors
    Remote work
    10 hours per week

    Auraone

    New York, NY
    4 days ago
  • Mercor is seeking experienced Nurse Practitioners to evaluate AI-generated clinical content and workflows, applying real-world expertise in diagnosing and managing patient care. This fully remote role focuses on evaluating AI-driven clinical assessments, treatment plans... 
    Remote job
    Weekly pay
    Immediate start

    Modern MedEd LLC

    New York, NY
    4 days ago
  • $37 per unit

     ...technical talent with leading AI research labs. Headquartered in...  ...Dorsey . Position: AI Safety Specialist Type: Contract...  ...Remote Role Responsibilities Evaluate the accuracy and depth of AI-...  ...with subject matter experts to ensure consistency, relevance... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    22 days ago
  • $100 - $120 per unit

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Baseball Fan – Live MLB Game AI Evaluator (US) Type: Contract Compensation:...  ..., schedules, broadcasts, highlights, and expert-style analysis. Run the same question... 
    Contract work
    Work experience placement
    Summer work
    Remote work

    Mercor

    New York, NY
    12 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    28 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    7 days ago
  • Joinkeywork is seeking a chemistry reviewer to validate AI-generated chemistry content. You will write problems, review peers, and assess mechanisms, stoichiometry, and procedures for safety. Strong written English and ability to justify conclusions are essential. You... 
    Remote work

    Sovrano AI

    New York, NY
    4 days ago
  • $85 per hour

    Process Safety Engineer AI Training Evaluator Review AI-generated content on EHS, process safety, and energy compliance using your professional experience. This remote, flexible project pays up to $85 per hour. Remote Hourly $85/hr $85/hr Compensation Contract Engagement... 
    Hourly pay
    Contract work
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    2 days ago
  • $68 per hour

    Registered Nurse AI Evaluation Expert (English) Use your nursing judgment to review AI patient education, triage guidance, clinical documentation...  ...train and assess AI systems. Your work focuses on accuracy, safety, clarity, and alignment with nursing practice. Review AI-... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    2 days ago
  •  ...the surgical setting. You will assess and implement anesthesia plans, monitor patients, and collaborate with physicians to ensure safety and quality. The role emphasizes compliance with HIPAA, regulatory standards, and continuous professional development. Prior CRNA experience... 
    Part time

    Highmark Health

    New York, NY
    3 days ago
  • Invisible Technologies Inc. seeks an AI QA trainer to harden model reasoning and reliability...  ...failure modes to raise standards for safety, bias checks, and prompt robustness. As a...  ...with researchers and engineers to evaluate LLM outputs, identify edge cases, and propose... 
    Remote job
    For contractors
    Flexible hours

    Invisible Technologies Inc.

    New York, NY
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Safety Practitioner - Expert Evaluator. Be the first to apply!