Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Conversation Evaluator - Remote

$70 per hour
Temporary

Recruitment Room

AI Conversation Evaluator - Remote

Employment Type: Contractor
Work Arrangement: Remote – Global
Language Requirement: English
Compensation: $45–$70 per hour
Estimated Time Commitment: Approximately 7.5 hours
Project Duration: November 15–18, 2026
Project Kickoff: November 15, 2026

ABOUT THE ROLE

We are seeking active AI users to participate in a short-term project evaluating the quality and usability of conversational AI assistants.

You will interact with different versions of AI assistants through realistic, everyday conversations, compare their responses, and provide structured ratings and written feedback.

The objective is to understand how natural, helpful, clear, and enjoyable different AI experiences are from the perspective of everyday users. Your feedback will contribute to improving conversational quality and overall user satisfaction.

No coding skills, specialist professional background, or previous AI evaluation experience are required.

KEY RESPONSIBILITIES

  • Engage in a series of natural, multi-turn conversations with different AI assistants.

  • Test everyday scenarios involving writing, planning, research, brainstorming, and general questions.

  • Compare responses from different AI experiences and identify your preferred output.

  • Evaluate conversational quality, helpfulness, clarity, tone, and overall user satisfaction.

  • Identify responses or behaviours that feel unnatural, repetitive, unnecessary, or unhelpful.

  • Provide concise written feedback highlighting positive and negative interaction experiences.

  • Follow testing guidelines and accurately document ratings and observations.

REQUIRED QUALIFICATIONS

  • Active ChatGPT user who accesses the platform at least three times per week.

  • Personal ChatGPT Free or Plus account.

  • Comfortable interacting with AI assistants for everyday tasks.

  • Strong attention to detail and ability to follow instructions.

  • Clear written communication skills.

  • English proficiency.

  • Access to a computer or smartphone.

  • Availability to participate during the November 15–18 project window.

PREFERRED QUALIFICATIONS

No specific professional background is required. Generalists from different industries and professional backgrounds are encouraged to participate.

Previous AI evaluation experience and coding skills are not required.

TECHNICAL REQUIREMENTS

Candidates must have:

  • An active personal ChatGPT account on either the Free or Plus plan.

  • Access to a computer or smartphone.

  • Reliable internet access to complete the evaluation activities.

PROJECT TIMELINE AND COMMITMENT

  • Project kickoff: November 15, 2026.

  • Expected completion: November 18, 2026.

  • Estimated total time commitment: Approximately 7.5 hours.

  • Short-term engagement with flexibility during the project window.

COMPENSATION

  • Hourly compensation: $45–$70.

  • Contractor engagement.

  • Payment is subject to completion of the assigned evaluation activities and applicable project requirements.

SELECTION PROCESS

  1. Application: Submit an application and confirm availability during the project window.

  2. Screening questions: Provide information about ChatGPT usage frequency and account type (Free or Plus).

  3. Interview: A short interview may be conducted to confirm eligibility.

  4. Profile review: Candidate profiles are reviewed for suitability.

  5. Onboarding: Selected candidates receive project instructions and setup guidance.

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the AI Conversation Evaluator - Remote in Remote vacancy
  • $20 - $160 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour Location: Remote Role Responsibilities Evaluate... 
    Remote work
    Contract work
    Summer work

    Mercor

    San Francisco, CA
    1 day ago
  •  ...Conversational AI Evaluator is a remote evaluation track for reviewing conversational ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  •  ...Tutor Conversation AI Evaluator is a remote review track for evaluating AI outputs across tutor conversation ai specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next... 
    Remote job
    Hourly pay
    For contractors
    Work experience placement
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  • $100 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Clinical Mental Health Expert — AI Conversation Evaluation Type: Contract Compensation: $100/hour Location: Remote Role Responsibilities Review pairs... 
    Remote work
    Contract work
    Summer work

    Mercor

    Houston, TX
    20 days ago
  •  ...About the role AI assistants are becoming part of everyday...  ...account, engage in everyday conversations, and share your feedback on what...  ...preferred experience. Evaluate conversational quality, helpfulness...  ...familiar with. Work remotely, with flexibility during the... 
    Remote work
    Hourly pay
    Temporary work
    Part time

    micro1

    San Francisco, CA
    7 days ago
  • $40 - $80 per hour

     ...Job Description Job Description Job Description: Paralegal – AI Training & Legal Evaluation Role Type: Contractor Location: Remote (Worldwide, U.S. Practice Experience Required) Weekly Commitment: Approximately 10+ Hours About the Role We are seeking experienced... 
    Remote job
    Hourly pay
    For contractors
    Worldwide
    10 hours per week
    Flexible hours

    YO AI Labs

    New York, NY
    3 days ago
  • $104k

     ...What We're Researching We're hiring AI evaluators to assess the accuracy and helpfulness of...  ...custom platform. As you analyze these conversations, you will pinpoint specific failures, logical...  ...response quality. This is an ongoing remote engagement requiring 20+ hours per week... 
    Remote work
    Hourly pay
    Contract work

    Terac

    Remote
    8 days ago
  •  ...Seeking PhD-level quantitative finance experts, the full-time Remote AI Research Evaluator will assess AI-generated financial content and contribute to the evaluation of advanced AI models in a flexible, remote contract role. Key Responsibilities Assessing the factuality... 
    Remote work
    Full time
    Contract work
    Flexible hours

    Virtual Vocations Inc

    United States
    14 hours ago
  • $14.5 per hour

    A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical... 
    Remote work
    Hourly pay
    Part time
    Flexible hours

    Welo Data

    United States
    19 hours ago
  • $14.5 per hour

     ...AI Web Search Evaluator As a Web Search Evaluator, you will play a key role in improving the quality of search engine results, ensuring users...  ...or data analysis is beneficial. Seeks a flexible, remote job that can easily fit around other commitments. Project... 
    Remote work
    Hourly pay
    Part time
    Currently hiring
    Immediate start
    Work from home
    10 hours per week
    Flexible hours

    Welo Data

    United States
    4 days ago
  • $14.5 per hour

     ...AI Web Search Evaluator Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant,...  ...suits you—choose your hours and achieve work-life balance. Remote Work: Enjoy the freedom of working from home anywhere in... 
    Remote work
    Bi-weekly pay
    Hourly pay
    Part time
    Immediate start
    Work from home
    Flexible hours

    Welo Data

    United States
    4 days ago
  •  ...Alignerr Corp. is seeking Creative Writing Evaluators to shape AI training by judging AI-generated writing across fiction, poetry, essays, and...  ..., voice, coherence, and reader engagement. Work is fully remote on an hourly contract, with 10–40 hours per week available.... 
    Remote work
    Hourly pay
    Contract work
    10 hours per week

    Alignerr Corp.

    United States
    3 days ago
  • A leading AI research firm is looking for a contractor in internal or emergency medicine to help enhance language models by providing...  ...The role requires a commitment of 20-40 hours per week in a fully remote environment, focusing on improving diagnostic accuracy in AI... 
    Remote job
    For contractors

    Turing

    Boston, MA
    3 days ago
  • $45k - $60k

    Aston Carter seeks an AI Design Evaluator / Design Quality Specialist to assess AI-generated design outputs and provide actionable feedback....  ...typography/layout knowledge, and excellent communication. This fully remote, permanent role is based out of Atlanta, GA, with a pay... 
    Remote job
    Permanent employment

    Aston Carter

    Atlanta, GA
    14 hours ago
  • $50 per hour

     ...Designers and UX Specialists to join our Expert Network to train and evaluate AI models using real design expertise. If selected, you’ll...  ...one hour of uninterrupted work, though shorter tasks exist. You can work remotely from home in a flexible schedule. #J-18808-Ljbffr... 
    Remote work
    Hourly pay
    Work from home
    Flexible hours

    Prolific Academic Ltd

    New York, NY
    19 hours ago
  • Prolific is seeking HR and Talent Professionals in San Jose, California to join their Expert Network. Responsibilities include evaluating AI-generated content and creating HR scenarios. Ideal candidates will have professional HR experience, strong compliance expertise,... 
    Remote job
    Hourly pay
    Work from home
    Flexible hours

    Prolific

    San Jose, CA
    2 days ago
  • $20 per hour

     ...Experience: 1 - 3 years Help improve leading AI models through research, reasoning and educational evaluation. This remote opportunity is for students and working...  ...established Gemini App account with genuine conversation history going back at least 18 months. ~ At... 
    Remote work
    Contract work
    Immediate start

    Braintrust

    San Francisco, CA
    23 days ago
  •  ...Evaluators review the answers an AI assistant gives to food delivery customers, judging quality, relevance and accuracy...  ..., suggest short fixes and write sample conversations that sound like real customers. The contract work is remote, and no prior AI experience is needed.... 
    Remote work
    Contract work

    micro1

    Remote
    3 days ago
  •  ...AI Policy Compliance AI Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios...  ...model defenses across single-turn and multi-turn conversations. Triage emerging attack vectors and route them to... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    9 days ago
  •  ...Description Job Description Job Title: Food Delivery AI Response Evaluator Role Type: Contractor Location: Remote Job Summary We are seeking detail-oriented...  ..., actionable feedback. # Create example conversations that reflect realistic food delivery customer... 
    Remote job
    For contractors

    YO AI Labs

    New York, NY
    8 days ago
  • $10 - $30 per hour

     ...Role Title: Food Delivery AI Response Evaluator Role Type : Contractor Location: Remote micro1 is engaging Food Delivery AI Response Evaluators to contribute...  ...improvement. Create illustrative example conversations that reflect authentic food delivery customer... 
    Remote work
    Contract work
    For contractors
    Shift work

    micro1

    United States
    9 days ago
  • $70 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Compensation: $70/hour Location: Remote Duration: 6 weeks Commitment:...  ...resume and application form. A 25-minute conversational interview covering your background,... 
    Remote work
    Contract work
    Summer work
    Immediate start

    Mercor

    San Francisco, CA
    13 days ago
  • Planet Pharma is seeking experienced clinicians to evaluate frontier AI models in real clinical work. You will design realistic tasks such as SOAP notes, reconciliations, triage decisions, discharge summaries, patient education, or protocols, and run them through frontier... 
    Remote job

    Planet Pharma

    San Francisco, CA
    2 days ago
  • Planet Pharma seeks experienced scientists and engineers to design and evaluate frontier AI models on real technical work, from data analysis to design verifications and reporting. Tasks are grounded in authentic practice and checked against professional standards, with... 
    Remote job

    Planet Pharma

    San Francisco, CA
    2 days ago
  • YO AI Labs seeks a former U.S. Consular Officer or Passport Specialist to evaluate AI-generated guidance on passport and citizenship matters. This remote, contractor role asks for 20-30 hours per week and leverages your State Department experience to ensure accuracy and... 
    Remote job
    For contractors

    YO AI Labs

    New York, NY
    4 days ago
  • Mercor is hiring a Healthcare Expert (contract, remote) to evaluate AI-generated medical responses and improve model reasoning. You will assess input data, compare outputs, and articulate reasons for superiority, focusing on clinical accuracy, safety, and sound medical... 
    Remote job
    Contract work
    Immediate start

    Mercor

    San Francisco, CA
    4 days ago
  • $21 - $65 per hour

    pofft Technology Inc. seeks photography professionals to support AI image quality evaluation on a remote contract basis within the United States. You will compare original photos with AI-generated images and provide detailed feedback to improve data quality for next‑generation... 
    Remote job
    Hourly pay
    Contract work
    For contractors

    pofft Technology Inc.

    Tallahassee, FL
    3 days ago
  • Turing Enterprises, Inc. in the San Francisco area is seeking a freelance journalist to help evaluate and improve AI using newsroom expertise, editorial judgment, and clear communication. No prior AI experience required. You will design and assess real-world editorial and... 
    Remote job
    Freelance

    Turing

    San Francisco, CA
    4 days ago
  • Ali Aydi is seeking photography professionals to support an AI data quality evaluation project. This remote contract role involves comparing original photographs with AI-generated images to identify inconsistencies and document feedback. Responsibilities include uploading... 
    Remote job
    Contract work

    Ali Aydi

    Tallahassee, FL
    3 days ago
  • $21 - $65 per hour

    EliteBrain is seeking photographers to remotely evaluate and improve visual data quality for AI systems. You will upload original photos, compare them with AI-generated images, and provide detailed feedback in clear English. The role is remote within the United States... 
    Remote job
    Hourly pay
    For contractors

    NEPSE Trading

    Tallahassee, FL
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Conversation Evaluator - Remote. Be the first to apply!