Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Support QA & Scenario Evaluator (Remote)

prolificacademicltd

New York, NY
  • Remote job

Prolific Academic Ltd is seeking individuals to help train and assess AI systems through paid, task-based work. Responsibilities include reviewing support conversations, drafting realistic customer tickets, and improving customer-facing guidance. A short skills assessment is part of the selection process, after which you can take on available tasks. Applicants should have professional experience in high-volume customer support and strong written English. #J-18808-Ljbffr prolificacademicltd

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the AI Support QA & Scenario Evaluator (Remote) in New York, NY vacancy
  • prolificacademicltd is offering a remote, flexible-role focused on evaluating and improving AI-generated customer support content. You will assess model replies, create test tickets and dialogue flows, and review FAQs to ensure clarity and accuracy. The position requires... 
    Remote job
    Flexible hours

    prolificacademicltd

    New York, NY
    2 days ago
  • Appsierra Group is seeking an experienced Customer Service Representative Expert to develop and evaluate realistic customer service scenarios used to train AI systems. The role is remote and contract-based, offering flexible, asynchronous work. You will create scenarios... 
    Remote job
    Contract work
    Flexible hours

    Recruitment Room

    New York, NY
    5 days ago
  •  ...seeking an experienced customer support professional to help train and assess AI systems. You will review generated...  ...content, create realistic customer scenarios, and judge whether responses meet...  ...sound support standards, working remotely on task-based assignments.... 
    Remote job

    prolificacademicltd

    New York, NY
    3 days ago
  •  ...a technically proficient customer support professional to help train and assess AI systems. The role involves reviewing...  ..., crafting realistic customer scenarios, and judging whether responses meet...  ...sound support standards. Tasks are remote and pay is task-based, following a... 
    Remote job

    prolificacademicltd

    New York, NY
    3 days ago
  • Role Title: AI Product Evaluator Role Type: Contractor Location: India, Japan or United Kingdom, Remote Our team is engaging expert AI Product Evaluators...  ...to detailed, pre-defined scenarios and guidelines. Execute a...  ...digital product, software QA, or AI research projects is... 
    Remote work
    For contractors

    Remotebridge

    New York, NY
    5 days ago
  • Remotebridge is seeking an AI Product Evaluator for remote, contract work across international teams. You...  ...AI models and agents against defined scenarios, document findings, and help ensure high...  ...within strict timeframes. Prior QA or AI research experience is a plus.... 
    Remote job
    Contract work

    Remotebridge

    New York, NY
    3 days ago
  • Mindrift is seeking US-licensed physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation project. You will review cases, score diagnostic...  ...contact or malpractice exposure. Contributors work remotely, choose tasks, and earn fixed per-task rates plus... 
    Remote job
    For contractors
    Flexible hours

    Toloka

    Annapolis, MD
    3 days ago
  • Appsierra Group is seeking an AI Session Annotator for a fully remote, independent contractor role. You will review recorded AI sessions, compare responses to visuals, and provide precise written feedback using a detailed rubric. Strong English, attention to detail, and... 
    Remote job
    Weekly pay
    For contractors
    Flexible hours

    Recruitment Room

    New York, NY
    5 days ago
  • $30 per hour

    Prolific Academic Ltd offers remote paid tasks to help improve AI systems by evaluating automated support conversations and reviewing generated content for accuracy and tone. Participants complete a brief skills test before being invited to join. Ideal contributors have... 
    Remote job
    Hourly pay

    prolificacademicltd

    New York, NY
    3 days ago
  •  ...Talent Acquisition experts to join our Expert Network and help train AI models using real-world HR expertise. You'll complete a brief...  ...paid to train AI models from home. Expect projects that involve evaluating HR outputs, creating realistic HR workflows, and annotating data... 
    Remote job
    Flexible hours

    Prolific

    New York, NY
    3 days ago
  • $40 per hour

    Prolific Academic Ltd. seeks a frontend quality specialist to assess AI-generated HTML and CSS. You will review rendered output, verify...  ..., styling quality, accessibility, and cross‑browser behavior. Remote flexibility is offered with compensation up to $40 per hour based... 
    Remote job
    Hourly pay

    prolificacademicltd

    New York, NY
    2 days ago
  • $8 - $65 per hour

    agency is seeking a remote contract contractor to evaluate and improve AI responses to coding tasks in Latin American Spanish. You will examine technical problem solving, verify explanations and code, and assess fluent, natural language usage. The role blends software evaluation... 
    Remote job
    Hourly pay
    Contract work
    For contractors

    agency

    New York, NY
    2 days ago
  • Inworld AI is seeking a voice evaluation specialist who will listen to paired or grouped speech samples, judge naturalness and accuracy, and help refine voices for real-time applications. This role is remote within the United States and involves structured reporting to... 
    Remote job

    Inworld AI

    New York, NY
    5 days ago
  • $80 per hour

    A leading AI consultancy is searching for QA analysts to validate AI agents in a project focused on improving...  .... This opportunity is a flexible, remote, part-time role ideal for curious thinkers...  ...and identify inconsistencies in AI evaluation. Competitive compensation up to $80/... 
    Remote job
    Part time
    Flexible hours

    Mindrift

    Jackson, MS
    5 days ago
  • $35 - $45 per hour

    ManpowerGroup seeks a Bilingual Linguist (Spanish/English) for expert evaluation of AI outputs. This remote, part-time role offers 20 hours per week with pay between $35 and $45 per hour (W2). A Mac computer running macOS 15 Sequoia or later is required. You will evaluate... 
    Remote job
    Hourly pay
    Part time

    Experis Technology Group

    New York, NY
    5 days ago
  • AI Trainer Jobs is seeking a Pharmacist for a remote review track evaluating AI outputs across pharmacist operations workflows. You will grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk and document next steps for modeling team training... 
    Remote job
    For contractors
    10 hours per week

    Auraone

    New York, NY
    4 days ago
  • Zekai is seeking an AI Product Evaluator for a 100% remote contractor role across global teams. You will apply expertise to train next-generation AI...  ...systems by evaluating models and agents against predefined scenarios. You will document findings, reset usage limits, and... 
    Remote job
    For contractors

    Zekai

    New York, NY
    2 days ago
  • AuraOne is seeking an Everyday AI Agent Tester specialist operations contractor for remote US-eligible work. You will review AI outputs against workflows, policy, and playbooks, grade tone and escalation, and document next steps for model training. Ideal candidates have... 
    Remote job
    For contractors

    Auraone

    New York, NY
    5 days ago
  •  ...Classroom Scenario AI Evaluator is a remote evaluation track for reviewing classroom scenario ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  • $30 per hour

     ...Academic Ltd seeks individuals to review AI-generated customer support replies for accuracy, tone, and...  ...systems by creating realistic customer scenarios and labeling sentiment to help improve AI responses. The role is remote with flexible hours and paid per task.... 
    Remote job
    Hourly pay
    Flexible hours

    prolificacademicltd

    New York, NY
    2 days ago
  • CNTXT, Inc. is seeking a fully remote hourly contractor to support AI data and language projects. You will work on prompts, voice samples, data annotation, LLM evaluation, and localization QA for American English content. The ideal candidate is a native American English... 
    Remote job
    Hourly pay
    For contractors

    CNTXT, Inc.

    New York, NY
    1 day ago
  • This is a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible...  ...support AI training datasets. LLM evaluation: reviewing AI-generated responses...  ...linguistic appropriateness. Localization QA: ensuring terminology, tone,... 
    Remote work
    Hourly pay
    For contractors
    Flexible hours

    CNTXT, Inc.

    New York, NY
    1 day ago
  • $30 per hour

     ...seeking a part-time independent contractor to evaluate AI shopping assistant interactions. The role...  ...commit to 20+ hours per week and work remotely on our internal platform. Payments are...  ...of $30 per hour. Prior experience in data QA for AI and e-commerce contexts is preferred... 
    Remote job
    Hourly pay
    Part time
    For contractors

    Terac

    New York, NY
    5 days ago
  • $60 per hour

    A leading AI company is seeking QAs for autonomous AI agents to ensure quality assurance in complex task evaluations. This remote project-based role is ideal for individuals who excel in analytical thinking and have experience in evaluating system logic. The opportunity... 
    Remote job
    Flexible hours

    Mindrift

    Madison, WI
    5 days ago
  • $20 - $40 per hour

     ...is seeking detail-oriented contributors to craft realistic AI task scenarios and evaluate agent performance. You will describe steps, judge outcomes...  ...to help developers improve automation. The role is remote, part-time, hourly, 10-40 hours per week, paid weekly in USD... 
    Remote job
    Hourly pay
    Weekly pay
    Part time
    10 hours per week

    Vettedlance

    New York, NY
    3 days ago
  • CNTXT AI is a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible hour basis. You will perform data annotation, LLM evaluation, content generation, and localization QA, applying senior-level European Portuguese localization... 
    Remote job
    Hourly pay
    For contractors
    Flexible hours

    CNTXT AI

    New York, NY
    3 days ago
  • Agency is seeking a contractor to help evaluate and improve an AI model’s software engineering knowledge. You will test coding tasks and CS questions...  ...failure cases to improve prompts and evaluation criteria. Remote contract work with rate based on experience. #J-18808-... 
    Remote job
    Contract work
    For contractors

    agency

    New York, NY
    3 days ago
  • $80 - $120 per hour

     ...technical talent with leading AI research labs. Headquartered in...  .../ trademark / copyright law Evaluator Type: Contract Compensation...  ...$80–$120/hour Location: Remote Role Responsibilities...  ...Submit form Resources & Support For details about the interview... 
    Remote work
    Contract work
    Summer work
    Work at office

    Mercor

    Chicago, IL
    10 days ago
  •  ...a Sales and Business Development Expert for a remote contractor role focused on an internal AI training project. You will bring your sales experience...  ...written and verbal communication. You will evaluate AI interview scenarios, simulate conversations across industries,... 
    Remote job
    For contractors

    Remotebridge

    New York, NY
    3 days ago
  • $25 - $35 per hour

     ...looking for a Professional Domain Expert (Real Estate and Leasing) to work part-time and remotely. Responsibilities include writing prompts for real estate scenarios, evaluating AI outputs, and providing evidence-based evaluations. The ideal candidate has expertise in real... 
    Remote job
    Hourly pay
    Part time

    Crossing Hurdles

    New York, NY
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Support QA & Scenario Evaluator (Remote). Be the first to apply!