Model Evaluator
$20 - $28 per hourTemporary
YPAI
Rate and compare AI answers and explain why, as paid work from home. It suits careful readers who can say why an answer is wrong, not only that it is.
What the work is for
Your ratings and written reasons help train AI models and test their answers.
The work
- Rate and compare AI answers against the project's criteria.
- Write a short, concrete reason for each rating. It matters as much as the rating itself.
- Check the facts in an answer. Rewrite weak answers where the task asks for it.
- Flag unsafe, false or policy-breaking output explicitly.
What to expect
- Each project has its own rating criteria and instructions for judging answers.
- You work in YPAI's own review console, where each project's rating criteria are set up.
You need
- Strong reading comprehension in the language you evaluate.
- The ability to explain why, not only what.
- 18 years or older.
Good to know
- You will read unsafe or offensive output at times. Judging it is part of the work.
- Work depends on the projects running in the language you evaluate. There can be gaps between projects.
Pay
$20–$28/hr. Rate shown is for the United States. This role is open in 12 countries, with local rates; pick your country on YPAI.
YPAI pays every two weeks for approved work via PayPal, Wise or bank transfer. Work is by invitation after your profile is reviewed.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Model Evaluator in Remote vacancy
- Table Image Reasoning Model Evaluator is a remote evaluation track for reviewing table image reasoning model evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback...SuggestedHourly payFor contractorsRemote work10 hours per week
- AI Trainer Jobs is seeking a remote contractor to evaluate people ops / recruiting prompts and responses against a evolving quality rubric. You will compare model outputs, label edge cases, and provide structured feedback for retraining. Responsibilities include evaluating...SuggestedRemote jobFor contractors
- ...Job Description Job Description Job Title: Language Model Evaluator Job Summary We are seeking a Language Model Evaluator to assess, benchmark, and improve the quality, accuracy, safety, and reliability of Large Language Models (LLMs) and Generative AI applications...SuggestedFull timeRemote work
- Mercor is seeking board-certified psychiatrists (MDs and DOs) to evaluate health AI models remotely in the United States. You will assess AI-generated psychiatric answers for accuracy and safety, reading de-identified patient charts. We are looking for active US licenses...SuggestedRemote jobPart timeImmediate startFlexible hours
- Mercor is seeking experienced MDs or DOs to evaluate health AI models remotely in the United States. You will review outputs across diagnoses, clinical reasoning and care workflows, and provide structured feedback to ensure medical accuracy and safety. The role requires...SuggestedRemote jobPart timeFor contractors
- ...Operations Research Model Evaluator is a remote review track for evaluating AI outputs across operations research model research review reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the...Remote jobHourly payFor contractors10 hours per week
- ...Frontier Model Misuse Risk Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team...Remote jobHourly payFor contractors10 hours per week
- ...Quant Finance Reasoning Model Evaluator is a remote review track for evaluating AI outputs across finance and risk workflows. Reviewers grade calculations, narrative reasoning, and policy adherence; flag compliance and reconciliation issues; and document the correct treatment...Remote jobHourly payFor contractorsWork experience placement10 hours per week
- ...Policy Preference Reward Model Evaluator is a remote review track for evaluating AI outputs in policy review workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can...Remote jobHourly payFor contractors10 hours per week
- ...Medical Document OCR Model Evaluator is a remote clinical-review track for evaluating AI outputs that touch clinical review. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning...Remote jobHourly payFor contractors10 hours per week
$85 per hour
...Location: Remote Role Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms , Kubernetes , CI/CD systems , observability...Contract workSummer workRemote work- ...st shift (United States of America)Please review the following job description:Responsible for leading one or more Financial Crimes model analytics, monitoring optimization, data strategy, or innovation functions within the Financial Intelligence Unit (FIU) related to Anti...Full timePart timeWork at officeRemote workShift workDay shift
- ...and deliver value to our business. At Applied Intuition, You Will: Conduct research on pretraining world-action foundation model with various world modalities including vision and physics associated with ego actions, serving the purpose for both robot action and...Full timeFor contractorsFor subcontractorCasual workInternshipWork at officeImmediate startRemote workDay shift
- ...We launched in January 2023 with the first human-like AI voice model. Today, we serve millions of users and thousands of businesses... ...relationships with image and video model providers. Identify and evaluate new partners and models to bring to ElevenCreative. Own the...Full timeImmediate startRemote work
- ...wholesale partners, and sells online through its website, which serves customers across the United States and globally at MEN'S FIT MODEL: This is a part-time opportunity to be a part of the design process and to influence the comfort and fit of our clothing. You will...Part timeFlexible hours
- ...We are hiring for: Family Model Provider Type: Regular If you are a positive and personable individual looking for a satisfying and fun opportunity to make a real difference in the lives of people with intellectual, developmental disabilities, and people facing...Full timeContract workFor contractorsLive inWork from home
- ...a caring, compassionate, and reliable person? Here is an opportunity to make a lasting impact on someone’s life – become an Family Model Provider !! This is a chance to help make a difference by opening your home to an individual with developmental and/or physical disabilities...Daily paid
$21 per hour
...up” sounds like a dream job… keep reading. We’re looking for a dynamic, camera-confident woman who loves creating content through modeling, styling videos, and connecting with women who want to feel amazing in their clothes. At Freckled Poppy, the country’s largest online...Hourly payFull timeMonday to FridayWeekend work$62.46k - $83.4k
...go to Workday and click "Find Jobs" link under Career Thank you for considering opportunities with us! Job Title Casualty Evaluator - Remote Requisition Number R8039 Casualty Evaluator - Remote (Open) Location Colorado - Home Teleworkers Additional Locations...Hourly payFull timeWork experience placementH1bWork at officeRemote workWork from homeFlexible hoursWeekend work- ...In conjuncion with Talent Model Recruiters, we are seeking new and experienced models for the apparel and fashion industry to display clothing and merchandise in commercials, advertisements, and/or fashion shows. Promote products and services in online ads, social media...Part timeFlexible hours
- ...Physician Services PC · Mental HealthValhalla, NYAllied Health Prof/TechnicalPer DiemAll ShiftsAs neededJob Summary: The Mental Health Evaluator (MHE) is a critical role dedicated to delivering comprehensive, clinically appropriate, culturally competent, and trauma-informed...
$250k - $320k
...committing to a multi-year, $100 million dollar trial? Today's models can contribute to these decisions, but they often miss a... ...environments and cultures in which decisions are made, and how to evaluate complex, multiplayer systems with many human and agent interactions...Full timeWork at officeLocal areaRemote workHome officeFlexible hours$260k
...clinical experts are included throughout the process. As a Clinical Evaluator, you'll work with our Clinical and Product teams to refine... ...working alongside AI-assisted review tools: critically evaluating a model's output and giving structured feedback that improves both the...Hourly payFull timeContract work10 hours per week- ...Description Mystery Evaluators (Gig / project-based work) - Freelance work - Multiple Opportunities Available in Tamil Nadu-Madurai, Salem, Vellore, Tiruppur, Coimbatore, Bhubaneshwar, Lucknow, Dehradun, Arunachal Pradesh, Andhra Pradesh, Kerala, Jammu, Varanasi...Extra incomeFreelance
$122.3k - $183.5k
...reliance and success.About the RoleAre you passionate about sharing leading practices and helping others avoid pitfalls? The Support Model & Governance Workday Success Plan Delivery team works directly with customers through strategic advisory consulting services,...Full timeWork at officeRemote workHome officeFlexible hours$200k - $230k
...tremendous career growth potential. Job Title: Foundation Model Engineer Location: 100% Remote (U.S.) Position Type: Full-... ...modern training stacks, careful dataset construction, rigorous evaluation methodology, and the engineering discipline to operate complex...Full timeH1bLocal areaImmediate startRemote workVisa sponsorship$185.12k - $197.6k
...behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Senior Advisor – Payment Model Evaluation based in the United States. We are seeking a highly experienced healthcare policy and evaluation expert to provide...Hourly payTemporary workPart timeRemote work10 hours per weekFlexible hours- Overview At PAM Health, we care for chronically and critically ill patients who require extended hospital care. PAM Health has over 80 hospital locations and employs over 11,000 people across the country. Our teams work together to deliver the highest level of compassionate...Full timeLocal area
- ...function secures Carnival Corp’s adoption of AI by ensuring GenAI, model, and agentic systems are deployed, validated, and operated... ...drive remediation.Design, build, and run model validation and evaluation harnesses to test AI models and pipelines for robustness, data...Full timePart timeWork at officeLocal areaWork from homeRelocationMonday to Thursday
$295k
...first enterprise AI company. We build cutting-edge foundation AI models and end-to-end products that are designed to solve real-world... ...Francisco, Montreal, Paris, Berlin and Seoul. Join us!Role Overview:Evaluation is critical to making progress in scaling intelligence. As...Full timeWork at officeLocal areaRemote workHome office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Model Evaluator. Be the first to apply!



