Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Safety Specialist - Evaluation Expert

$60 - $70 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: AI Safety Practitioner
Type: Contract
Compensation: $60–$70/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
  • Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
  • Apply and refine evaluation rubrics for RLHF , SFT , and AI safety benchmarking .
  • Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
  • Provide structured feedback to improve model alignment and safety performance.
  • Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.

Qualifications

Must-Have

  • Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline.
  • 5+ years of professional experience in AI Safety , Trust & Safety, journalism, public policy, scientific research, security, or a related field.
  • Excellent written English , critical thinking, and analytical reasoning skills.
  • Ability to consistently evaluate nuanced and policy-sensitive scenarios.

Preferred

  • Experience with AI Safety , RLHF , SFT , Trust & Safety, or AI evaluation.
  • Familiarity with safety policies, content moderation, or evaluation rubric development.
  • Experience reviewing complex, high-risk, or ambiguous content.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 11 days ago
Similar jobs that could be interesting for youBased on the AI Safety Specialist - Evaluation Expert in New York, NY vacancy
  • Invisible Technologies Inc. seeks an AI QA trainer to harden model reasoning and reliability...  ...failure modes to raise standards for safety, bias checks, and prompt robustness. As a...  ...with researchers and engineers to evaluate LLM outputs, identify edge cases, and propose... 
    Suggested
    Remote job
    For contractors
    Flexible hours

    Invisible Technologies Inc.

    New York, NY
    13 hours ago
  • $37 per unit

     ...technical talent with leading AI research labs. Headquartered...  ...Jack Dorsey . Position: AI Safety Specialist Type: Contract...  ...Role Responsibilities Evaluate the accuracy and depth of AI...  ...Collaborate with subject matter experts to ensure consistency, relevance... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    22 days ago
  • $35 - $42 per hour

     ...project availability Help Shape the Future of AI in Finance, Audit & Risk Volga Partners...  ...join our growing network ofon-call AI Evaluation Specialistssupporting the development, training...  ...’ on-call network of subject matter experts. As client projects become available,... 
    Suggested
    Hourly pay
    Extra income
    Permanent employment
    Contract work
    Temporary work
    Remote work
    Flexible hours

    Volga Partners

    New York, NY
    4 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    9 days ago
  •  ...Position Title: Senior Computer Programmer – AI Evaluation Specialist Location: Remote (Continental U.S.) Company: JR Software Solutions...  ...an AI Evaluation Specialist to determine the effectiveness, safety, usability, and readiness of enterprise AI tools for adoption... 
    Suggested
    Permanent employment
    Full time
    Remote work

    LinkedIn

    New York, NY
    13 hours ago
  • $68 per hour

    Registered Nurse AI Evaluation Expert (English) Use your nursing judgment to review AI patient education, triage guidance, clinical documentation...  ...train and assess AI systems. Your work focuses on accuracy, safety, clarity, and alignment with nursing practice. Review AI-... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    2 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...and Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    28 days ago
  •  ...education and research sector is seeking a Social Sciences PhD Expert for a remote part-time role, requiring strong analytical...  ...The position involves creating historically relevant prompts, evaluating AI outputs, and contributing to research initiatives. Ideal candidates... 
    Part time
    Remote work
    Flexible hours

    Crossing Hurdles

    New York, NY
    4 days ago
  • $8 - $65 per hour

     ...Are you an Arabic (Levantine) language expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into powerful...  ...and suggest improvements to our prompt engineering and evaluation metrics. A Master's degree in Linguistics, Middle... 
    Hourly pay
    Contract work
    For contractors
    Freelance
    Remote work

    Agency.com

    New York, NY
    3 days ago
  • $8 - $65 per hour

     ...Arabic (Levantine) Language Expert - Freelance AI Trainer Project World Wide - Remote Are you an...  ...looking for Arabic (Levantine) language specialists who live and breathe phonetics,...  ...improvements to our prompt engineering and evaluation metrics. A Master’s degree in... 
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start
    Remote work

    Meridial

    New York, NY
    13 hours ago
  • $55 per hour

     ...Freelance Biology Expert with Python - AI Trainer 5 days ago – Be among the first 25 applicants...  ...Do The Mindrift platform connects specialists with AI projects from major tech innovators...  ...comprehensive scoring criteria to evaluate the accuracy of the AI's answers.... 
    Part time
    Freelance
    Remote work

    Mind Rift

    New York, NY
    1 day ago
  •  ...we do at UNICEF. For every child, the right to safety How can you make a difference? The UNICEF Evaluation Office (EO) is commissioning a global evaluation...  ...effectively in the future. The EO is seeking a Senior Expert in Organizational Decision-making for this... 
    Remote job
    Work experience placement
    Work at office

    UNICEF

    New York, NY
    10 days ago
  • $30 per hour

    AI Safety & Evaluation Specialist Role type: Contractor Location: United States Commitment: Part-time, project-based Duration: 6 weeks Compensation: $30 $60 USD/hour Required Skills privacy risk assessment data protection ai evaluation... 
    Full time
    Part time
    For contractors

    Pro Integrate

    New York, NY
    7 days ago
  •  ...Job Description: Computational Biology & Bioinformatics Expert (PhD) Role Type: Contractor Location: Remote (Worldwide...  ...their scientific expertise to the development and evaluation of next-generation AI systems. In this role, you will apply your knowledge of... 
    Remote job
    For contractors
    Worldwide

    YO AI Labs

    New York, NY
    5 days ago
  • $8 - $65 per hour

    Antibiotic Resistance Specialist - Freelance AI Trainer Project United States of America Are you a master of antibiotic resistance eager to shape...  ..., and suggest improvements to our prompt engineering and evaluation metrics. A PhD in microbiology, molecular biology, or a... 
    Remote job
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start

    Meridial

    New York, NY
    2 days ago
  • $55 per hour

    Help evaluate AI responses and create biology prompts as a remote Biology Expert AI Trainer. The role is open to US candidates with graduate-level biology expertise and pays up to $55 per hour. Remote Hourly · $55/hr $55/hr Compensation Contract, part-time Engagement... 
    Hourly pay
    Contract work
    Part time
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    1 day ago
  • $8 - $65 per hour

    Agency seeks an Arabic (Levantine) language expert to shape AI training data and evaluate model performance. You will interact with the model in Levantine Arabic, document failures, and help improve prompt engineering and evaluation metrics. A Master's degree is ideal;... 
    Remote job
    Hourly pay
    Contract work
    For contractors

    Agency

    New York, NY
    4 days ago
  • $40 per hour

    Prolific is seeking Neuroscience Experts to act as evaluators for a high-level AI verification project. You will review dense scientific literature to judge the accuracy and structural integrity of AI-generated graphical abstracts, ensuring that summaries faithfully reflect... 
    Remote job

    Prolific

    New York, NY
    2 days ago
  • Agency is seeking a highly skilled Māori language specialist to help train large-scale AI models. You will review, annotate, and refine...  ...you’ll work remotely from the United States, evaluating model performance, providing expert feedback, and helping shape linguistic... 
    Remote job
    For contractors

    Agency

    New York, NY
    4 days ago
  • $40 per hour

     ...create high-quality STEM training data for frontier AI models. Our data is used directly in training and evaluation pipelines at leading AI labs to improve model reasoning in technical domains. We are looking for experts in Physics to design rigorous, deterministic... 
    Contract work
    Part time
    Remote work

    Anyone AI Inc.

    New York, NY
    4 days ago
  • $55 per hour

     ...collective intelligence to ethically shape the future of AI. What We Do The Mindrift platform connects specialists with AI projects from major tech innovators. Our...  ...AI. Define comprehensive scoring criteria to evaluate the accuracy of the AI’s answers. Correct the... 
    Part time
    Freelance
    Remote work

    Mindrift

    New York, NY
    4 days ago
  • $8 - $65 per hour

    Agency seeks a Maltese Language Specialist - AI Trainer for a remote contract role. You will annotate Maltese content, evaluate AI outputs for accuracy, and provide linguistic guidance to improve model performance. Responsibilities include identifying error patterns, refining... 
    Remote job
    Hourly pay
    Contract work

    Agency

    New York, NY
    4 days ago
  • $24 per hour

     ...Use your accounting and QuickBooks experience to help improve AI responses to real small-business and financial questions. In this remote freelance project, you'll evaluate accuracy, identify gaps and help build reliable standards for accounting-related AI tasks. A part... 
    Part time
    Freelance
    Remote work

    Braintrust

    New York, NY
    4 days ago
  • $110 - $140 per hour

     ...Bring senior chemistry expertise to AI evaluation projects focused on reaction mechanisms, analytical methods, lab safety, and chemical evidence. Review model responses and...  ...evidence. Pay $110–$140/hr Alpheva is a new expert network (launched September 2026); payout methods... 
    Hourly pay
    Contract work
    Remote work

    Alpheva

    New York, NY
    2 days ago
  • Handshake is looking for skilled QGIS users to support AI research through flexible, hourly contract work. You'll draw on hands-on GIS experience to evaluate AI-generated content and provide feedback that helps AI better understand GIS tasks and spatial data workflows.... 
    Remote job
    Hourly pay
    Contract work
    Flexible hours

    Handshake

    New York, NY
    2 days ago
  •  ...A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities... 
    Immediate start

    Crossing Hurdles

    New York, NY
    4 days ago
  • $65 - $70 per hour

     ...Biology Expert (PhD) — AI Safety is a remote red-team track for stress-testing AI systems against...  ...team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before...  ...evaluation Application boundary Creating a specialist profile records your experience and... 
    For contractors
    Remote work
    10 hours per week

    AuraOne

    New York, NY
    1 day ago
  •  ...Biology Subject Matter Expert (AI Training) About the Role We're looking for biology experts to help evaluate and improve AI systems trained on advanced scientific content. Your deep knowledge of biology, biochemistry, or biotechnology will directly shape how... 
    Hourly pay
    Ongoing contract
    Contract work
    Freelance
    Remote work
    Flexible hours

    Alignerr

    New York, NY
    4 days ago
  • $8 - $65 per hour

     ...Prolific is hiring Mental Health Professionals in New York to train and evaluate AI models. As a Domain Expert, you will be responsible for reviewing AI-generated responses, completing tasks related to psychology, and improving AI models based on your expertise. Pay rates... 
    Hourly pay
    Remote work
    Work from home
    Flexible hours

    Prolific

    New York, NY
    1 day ago
  • $85 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Jack Dorsey . Position: Fab Safety and Emergency Response Expert Type: Contract Compensation...  ..., and safety documentation. Evaluate the accuracy and depth of safety protocols... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Safety Specialist - Evaluation Expert. Be the first to apply!