Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Model Evaluator

$20 - $28 per hour
Temporary

YPAI

Rate and compare AI answers and explain why, as paid work from home. It suits careful readers who can say why an answer is wrong, not only that it is.

What the work is for

Your ratings and written reasons help train AI models and test their answers.

The work

  • Rate and compare AI answers against the project's criteria.
  • Write a short, concrete reason for each rating. It matters as much as the rating itself.
  • Check the facts in an answer. Rewrite weak answers where the task asks for it.
  • Flag unsafe, false or policy-breaking output explicitly.

What to expect

  • Each project has its own rating criteria and instructions for judging answers.
  • You work in YPAI's own review console, where each project's rating criteria are set up.

You need

  • Strong reading comprehension in the language you evaluate.
  • The ability to explain why, not only what.
  • 18 years or older.

Good to know

  • You will read unsafe or offensive output at times. Judging it is part of the work.
  • Work depends on the projects running in the language you evaluate. There can be gaps between projects.

Pay

$20–$28/hr. Rate shown is for the United States. This role is open in 12 countries, with local rates; pick your country on YPAI.

YPAI pays every two weeks for approved work via PayPal, Wise or bank transfer. Work is by invitation after your profile is reviewed.

Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Model Evaluator in Remote vacancy
  • Table Image Reasoning Model Evaluator is a remote evaluation track for reviewing table image reasoning model evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback... 
    Suggested
    Hourly pay
    For contractors
    Remote work
    10 hours per week

    AuraOne, Inc.

    New York, NY
    1 day ago
  • AI Trainer Jobs is seeking a remote contractor to evaluate people ops / recruiting prompts and responses against a evolving quality rubric. You will compare model outputs, label edge cases, and provide structured feedback for retraining. Responsibilities include evaluating... 
    Suggested
    Remote job
    For contractors

    AuraOne

    New York, NY
    15 hours ago
  •  ...Job Description Job Description Job Title: Language Model Evaluator Job Summary We are seeking a Language Model Evaluator to assess, benchmark, and improve the quality, accuracy, safety, and reliability of Large Language Models (LLMs) and Generative AI applications... 
    Suggested
    Full time
    Remote work

    Ova Technologies

    New York, NY
    2 days ago
  • Mercor is seeking board-certified psychiatrists (MDs and DOs) to evaluate health AI models remotely in the United States. You will assess AI-generated psychiatric answers for accuracy and safety, reading de-identified patient charts. We are looking for active US licenses... 
    Suggested
    Remote job
    Part time
    Immediate start
    Flexible hours

    Benture

    New York, NY
    3 days ago
  • Mercor is seeking experienced MDs or DOs to evaluate health AI models remotely in the United States. You will review outputs across diagnoses, clinical reasoning and care workflows, and provide structured feedback to ensure medical accuracy and safety. The role requires... 
    Suggested
    Remote job
    Part time
    For contractors

    Benture

    New York, NY
    15 hours ago
  •  ...Operations Research Model Evaluator is a remote review track for evaluating AI outputs across operations research model research review reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    21 days ago
  •  ...Frontier Model Misuse Risk Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  •  ...Quant Finance Reasoning Model Evaluator is a remote review track for evaluating AI outputs across finance and risk workflows. Reviewers grade calculations, narrative reasoning, and policy adherence; flag compliance and reconciliation issues; and document the correct treatment... 
    Remote job
    Hourly pay
    For contractors
    Work experience placement
    10 hours per week

    AuraOne Human Data

    Remote
    22 days ago
  •  ...Policy Preference Reward Model Evaluator is a remote review track for evaluating AI outputs in policy review workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    22 days ago
  •  ...Medical Document OCR Model Evaluator is a remote clinical-review track for evaluating AI outputs that touch clinical review. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  • $85 per hour

     ...Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms , Kubernetes , CI/CD systems , observability... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    15 days ago
  •  ...st shift (United States of America)Please review the following job description:Responsible for leading one or more Financial Crimes model analytics, monitoring optimization, data strategy, or innovation functions within the Financial Intelligence Unit (FIU) related to Anti... 
    Full time
    Part time
    Work at office
    Remote work
    Shift work
    Day shift

    Truist

    Charlotte, NC
    15 hours ago
  •  ...and deliver value to our business. At Applied Intuition, You Will: Conduct research on pretraining world-action foundation model with various world modalities including vision and physics associated with ego actions, serving the purpose for both robot action and... 
    Full time
    For contractors
    For subcontractor
    Casual work
    Internship
    Work at office
    Immediate start
    Remote work
    Day shift

    Applied Intuition

    Sunnyvale, CA
    2 days ago
  •  ...We launched in January 2023 with the first human-like AI voice model. Today, we serve millions of users and thousands of businesses...  ...relationships with image and video model providers. Identify and evaluate new partners and models to bring to ElevenCreative. Own the... 
    Full time
    Immediate start
    Remote work

    ElevenLabs

    Remote
    3 days ago
  •  ...wholesale partners, and sells online through its website, which serves customers across the United States and globally at MEN'S FIT MODEL: This is a part-time opportunity to be a part of the design process and to influence the comfort and fit of our clothing. You will... 
    Part time
    Flexible hours

    The Normal Brand

    Saint Louis, MO
    4 days ago
  •  ...We are hiring for: Family Model Provider Type: Regular If you are a positive and personable individual looking for a satisfying and fun opportunity to make a real difference in the lives of people with intellectual, developmental disabilities, and people facing... 
    Full time
    Contract work
    For contractors
    Live in
    Work from home

    RHA Health Services

    Jackson, TN
    3 days ago
  •  ...a caring, compassionate, and reliable person? Here is an opportunity to make a lasting impact on someone’s life – become an Family Model Provider !! This is a chance to help make a difference by opening your home to an individual with developmental and/or physical disabilities... 
    Daily paid

    Community Options

    Chattanooga, TN
    3 days ago
  • $21 per hour

     ...up” sounds like a dream job… keep reading. We’re looking for a dynamic, camera-confident woman who loves creating content through modeling, styling videos, and connecting with women who want to feel amazing in their clothes. At Freckled Poppy, the country’s largest online... 
    Hourly pay
    Full time
    Monday to Friday
    Weekend work

    ADP

    Hollywood, MD
    3 days ago
  • $62.46k - $83.4k

     ...go to Workday and click "Find Jobs" link under Career Thank you for considering opportunities with us! Job Title Casualty Evaluator - Remote Requisition Number R8039 Casualty Evaluator - Remote (Open) Location Colorado - Home Teleworkers Additional Locations... 
    Hourly pay
    Full time
    Work experience placement
    H1b
    Work at office
    Remote work
    Work from home
    Flexible hours
    Weekend work

    CSAA Insurance Group, a AAA Insurer

    Oregon State
    6 days ago
  •  ...In conjuncion with Talent Model Recruiters, we are seeking new and experienced models for the apparel and fashion industry to display clothing and merchandise in commercials, advertisements, and/or fashion shows. Promote products and services in online ads, social media... 
    Part time
    Flexible hours

    Caring Angels Home Care, LLC

    Haverstraw, NY
    3 days ago
  •  ...Physician Services PC · Mental HealthValhalla, NYAllied Health Prof/TechnicalPer DiemAll ShiftsAs neededJob Summary: The Mental Health Evaluator (MHE) is a critical role dedicated to delivering comprehensive, clinically appropriate, culturally competent, and trauma-informed... 

    HealthAlliance of the Hudson Valley

    Valhalla, NY
    4 days ago
  • $250k - $320k

     ...committing to a multi-year, $100 million dollar trial? Today's models can contribute to these decisions, but they often miss a...  ...environments and cultures in which decisions are made, and how to evaluate complex, multiplayer systems with many human and agent interactions... 
    Full time
    Work at office
    Local area
    Remote work
    Home office
    Flexible hours

    Elicit

    Remote
    12 days ago
  • $260k

     ...clinical experts are included throughout the process. As a Clinical Evaluator, you'll work with our Clinical and Product teams to refine...  ...working alongside AI-assisted review tools: critically evaluating a model's output and giving structured feedback that improves both the... 
    Hourly pay
    Full time
    Contract work
    10 hours per week

    Headway

    Remote
    15 days ago
  •  ...Description Mystery Evaluators (Gig / project-based work) - Freelance work - Multiple Opportunities Available in Tamil Nadu-Madurai, Salem, Vellore, Tiruppur, Coimbatore, Bhubaneshwar, Lucknow, Dehradun, Arunachal Pradesh, Andhra Pradesh, Kerala, Jammu, Varanasi... 
    Extra income
    Freelance

    BARE International

    Remote
    13 days ago
  • $122.3k - $183.5k

     ...reliance and success.About the RoleAre you passionate about sharing leading practices and helping others avoid pitfalls? The Support Model & Governance Workday Success Plan Delivery team works directly with customers through strategic advisory consulting services,... 
    Full time
    Work at office
    Remote work
    Home office
    Flexible hours

    Workday

    Salt Lake City, UT
    3 days ago
  • $200k - $230k

     ...tremendous career growth potential. Job Title: Foundation Model Engineer Location: 100% Remote (U.S.) Position Type: Full-...  ...modern training stacks, careful dataset construction, rigorous evaluation methodology, and the engineering discipline to operate complex... 
    Full time
    H1b
    Local area
    Immediate start
    Remote work
    Visa sponsorship

    Bright Vision Technologies

    Remote
    6 days ago
  • $185.12k - $197.6k

     ...behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Advisor – Payment Model Evaluation based in the United States. We are seeking a highly experienced healthcare policy and evaluation expert to provide... 
    Hourly pay
    Temporary work
    Part time
    Remote work
    10 hours per week
    Flexible hours

    Jobgether

    Remote
    3 days ago
  • Overview At PAM Health, we care for chronically and critically ill patients who require extended hospital care. PAM Health has over 80 hospital locations and employs over 11,000 people across the country. Our teams work together to deliver the highest level of compassionate...
    Full time
    Local area

    PAM Health

    Humble, TX
    15 hours ago
  •  ...function secures Carnival Corp’s adoption of AI by ensuring GenAI, model, and agentic systems are deployed, validated, and operated...  ...drive remediation.Design, build, and run model validation and evaluation harnesses to test AI models and pipelines for robustness, data... 
    Full time
    Part time
    Work at office
    Local area
    Work from home
    Relocation
    Monday to Thursday

    Carnival Cruise Line

    Miami, FL
    4 days ago
  • $295k

     ...first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world...  ...Francisco, Montreal, Paris, Berlin and Seoul. Join us!Role Overview:Evaluation is critical to making progress in scaling intelligence. As... 
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Model Evaluator. Be the first to apply!