Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Conversation Evaluator - Remote

$40 - $80 per hour

YO AI Labs

New York, NY
  • Remote job

Job Description

Job Description

Job Description: Paralegal – AI Training & Legal Evaluation

Role Type: Contractor
Location: Remote (Worldwide, U.S. Practice Experience Required)
Weekly Commitment: Approximately 10+ Hours

About the Role

We are seeking experienced paralegals to contribute their legal expertise to an AI training project. You will apply your practical knowledge of legal research, document review, citation verification, and document preparation to evaluate realistic legal tasks and improve AI systems.

Prior AI experience is not required.

Key Responsibilities
  1. Complete legal tasks involving document review, research, citation verification, and document preparation.
  2. Explain the reasoning behind legal decisions and assessments.
  3. Follow established review protocols and flag cases that require further clarification.
  4. Identify ambiguous issues requiring attorney judgment.
  5. Review other contributors' work for accuracy and consistency.
  6. Deliver high-quality work within assigned deadlines.
Required Qualifications
  1. At least 5 years of hands-on paralegal experience in litigation, corporate, or regulatory practice.
  2. Practical experience in document review, legal research, and citation verification.
  3. Paralegal certificate, bachelor's degree, or equivalent professional experience.
  4. Ability to manage multiple documents and identify inconsistencies.
  5. Strong written and verbal English communication skills.
  6. Experience handling complex legal assignments with accuracy and attention to detail.
Preferred Qualifications
  1. Experience with legal data projects, e-discovery platforms, or structured review protocols.
  2. Familiarity with AI or LLM evaluation.
  3. Strong analytical, organizational, and problem-solving skills.
Compensation & Engagement
  • $40–$80 per hour.
  • Approximately 10+ hours per week.
  • Flexible weekly schedule.
  • Remote contractor opportunity open worldwide to candidates with U.S. legal practice experience.
Selection Process
  1. Submit an application and complete screening questions.
  2. Complete a role-specific AI interview.
  3. Undergo profile review.
  4. Complete onboarding with the Talent Operations team.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the AI Conversation Evaluator - Remote in New York, NY vacancy
  • $20 - $160 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour Location: Remote Role Responsibilities Evaluate... 
    Remote work
    Contract work
    Summer work

    Mercor

    San Francisco, CA
    1 day ago
  •  ...Conversational AI Evaluator is a remote evaluation track for reviewing conversational ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  • $70 per hour

     ...AI Conversation Evaluator - Remote Employment Type: Contractor Work Arrangement: Remote – Global Language Requirement: English Compensation: $45–$70 per hour Estimated Time Commitment: Approximately 7.5 hours Project Duration: November 15–18, 202... 
    Remote work
    Hourly pay
    Contract work
    Temporary work
    For contractors

    Recruitment Room

    Remote
    3 days ago
  •  ...Tutor Conversation AI Evaluator is a remote review track for evaluating AI outputs across tutor conversation ai specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next... 
    Remote job
    Hourly pay
    For contractors
    Work experience placement
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  • $100 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Clinical Mental Health Expert — AI Conversation Evaluation Type: Contract Compensation: $100/hour Location: Remote Role Responsibilities Review pairs... 
    Remote work
    Contract work
    Summer work

    Mercor

    Houston, TX
    20 days ago
  •  ...About the role AI assistants are becoming part of everyday...  ...account, engage in everyday conversations, and share your feedback on what...  ...preferred experience. Evaluate conversational quality, helpfulness...  ...familiar with. Work remotely, with flexibility during the... 
    Remote work
    Hourly pay
    Temporary work
    Part time

    micro1

    San Francisco, CA
    7 days ago
  • $104k

     ...What We're Researching We're hiring AI evaluators to assess the accuracy and helpfulness of...  ...custom platform. As you analyze these conversations, you will pinpoint specific failures, logical...  ...response quality. This is an ongoing remote engagement requiring 20+ hours per week... 
    Remote work
    Hourly pay
    Contract work

    Terac

    Remote
    8 days ago
  •  ...Seeking PhD-level quantitative finance experts, the full-time Remote AI Research Evaluator will assess AI-generated financial content and contribute to the evaluation of advanced AI models in a flexible, remote contract role. Key Responsibilities Assessing the factuality... 
    Remote work
    Full time
    Contract work
    Flexible hours

    Virtual Vocations Inc

    United States
    13 hours ago
  • $14.5 per hour

    A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical... 
    Remote work
    Hourly pay
    Part time
    Flexible hours

    Welo Data

    United States
    19 hours ago
  • $14.5 per hour

     ...AI Web Search Evaluator As a Web Search Evaluator, you will play a key role in improving the quality of search engine results, ensuring users...  ...or data analysis is beneficial. Seeks a flexible, remote job that can easily fit around other commitments. Project... 
    Remote work
    Hourly pay
    Part time
    Currently hiring
    Immediate start
    Work from home
    10 hours per week
    Flexible hours

    Welo Data

    United States
    4 days ago
  • $14.5 per hour

     ...AI Web Search Evaluator Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant,...  ...suits you—choose your hours and achieve work-life balance. Remote Work: Enjoy the freedom of working from home anywhere in... 
    Remote work
    Bi-weekly pay
    Hourly pay
    Part time
    Immediate start
    Work from home
    Flexible hours

    Welo Data

    United States
    4 days ago
  •  ...Alignerr Corp. is seeking Creative Writing Evaluators to shape AI training by judging AI-generated writing across fiction, poetry, essays, and...  ..., voice, coherence, and reader engagement. Work is fully remote on an hourly contract, with 10–40 hours per week available.... 
    Remote work
    Hourly pay
    Contract work
    10 hours per week

    Alignerr Corp.

    United States
    3 days ago
  • A leading AI research firm is looking for a contractor in internal or emergency medicine to help enhance language models by providing...  ...The role requires a commitment of 20-40 hours per week in a fully remote environment, focusing on improving diagnostic accuracy in AI... 
    Remote job
    For contractors

    Turing

    Boston, MA
    3 days ago
  • $45k - $60k

    Aston Carter seeks an AI Design Evaluator / Design Quality Specialist to assess AI-generated design outputs and provide actionable feedback....  ...typography/layout knowledge, and excellent communication. This fully remote, permanent role is based out of Atlanta, GA, with a pay... 
    Remote job
    Permanent employment

    Aston Carter

    Atlanta, GA
    13 hours ago
  • $50 per hour

     ...Designers and UX Specialists to join our Expert Network to train and evaluate AI models using real design expertise. If selected, you’ll...  ...one hour of uninterrupted work, though shorter tasks exist. You can work remotely from home in a flexible schedule. #J-18808-Ljbffr... 
    Remote work
    Hourly pay
    Work from home
    Flexible hours

    Prolific Academic Ltd

    New York, NY
    18 hours ago
  • Prolific is seeking HR and Talent Professionals in San Jose, California to join their Expert Network. Responsibilities include evaluating AI-generated content and creating HR scenarios. Ideal candidates will have professional HR experience, strong compliance expertise,... 
    Remote job
    Hourly pay
    Work from home
    Flexible hours

    Prolific

    San Jose, CA
    2 days ago
  • $20 per hour

     ...Experience: 1 - 3 years Help improve leading AI models through research, reasoning and educational evaluation. This remote opportunity is for students and working...  ...established Gemini App account with genuine conversation history going back at least 18 months. ~ At... 
    Remote work
    Contract work
    Immediate start

    Braintrust

    San Francisco, CA
    23 days ago
  •  ...Evaluators review the answers an AI assistant gives to food delivery customers, judging quality, relevance and accuracy...  ..., suggest short fixes and write sample conversations that sound like real customers. The contract work is remote, and no prior AI experience is needed.... 
    Remote work
    Contract work

    micro1

    Remote
    3 days ago
  •  ...Secure Code Review AI Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack...  ...Score model defenses across single-turn and multi-turn conversations. Triage emerging attack vectors and route them to the... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  •  ...Description Job Description Job Title: Food Delivery AI Response Evaluator Role Type: Contractor Location: Remote Job Summary We are seeking detail-oriented...  ..., actionable feedback. # Create example conversations that reflect realistic food delivery customer... 
    Remote job
    For contractors

    YO AI Labs

    New York, NY
    8 days ago
  • $10 - $30 per hour

     ...Role Title: Food Delivery AI Response Evaluator Role Type : Contractor Location: Remote micro1 is engaging Food Delivery AI Response Evaluators to contribute...  ...improvement. Create illustrative example conversations that reflect authentic food delivery customer... 
    Remote work
    Contract work
    For contractors
    Shift work

    micro1

    United States
    9 days ago
  • $70 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Compensation: $70/hour Location: Remote Duration: 6 weeks Commitment:...  ...resume and application form. A 25-minute conversational interview covering your background,... 
    Remote work
    Contract work
    Summer work
    Immediate start

    Mercor

    San Francisco, CA
    13 days ago
  • Planet Pharma is seeking experienced clinicians to evaluate frontier AI models in real clinical work. You will design realistic tasks such as SOAP notes, reconciliations, triage decisions, discharge summaries, patient education, or protocols, and run them through frontier... 
    Remote job

    Planet Pharma

    San Francisco, CA
    2 days ago
  • Planet Pharma seeks experienced scientists and engineers to design and evaluate frontier AI models on real technical work, from data analysis to design verifications and reporting. Tasks are grounded in authentic practice and checked against professional standards, with... 
    Remote job

    Planet Pharma

    San Francisco, CA
    2 days ago
  • YO AI Labs seeks a former U.S. Consular Officer or Passport Specialist to evaluate AI-generated guidance on passport and citizenship matters. This remote, contractor role asks for 20-30 hours per week and leverages your State Department experience to ensure accuracy and... 
    Remote job
    For contractors

    YO AI Labs

    New York, NY
    4 days ago
  • Mercor is hiring a Healthcare Expert (contract, remote) to evaluate AI-generated medical responses and improve model reasoning. You will assess input data, compare outputs, and articulate reasons for superiority, focusing on clinical accuracy, safety, and sound medical... 
    Remote job
    Contract work
    Immediate start

    Mercor

    San Francisco, CA
    4 days ago
  • $21 - $65 per hour

    pofft Technology Inc. seeks photography professionals to support AI image quality evaluation on a remote contract basis within the United States. You will compare original photos with AI-generated images and provide detailed feedback to improve data quality for next‑generation... 
    Remote job
    Hourly pay
    Contract work
    For contractors

    pofft Technology Inc.

    Tallahassee, FL
    3 days ago
  • Turing Enterprises, Inc. in the San Francisco area is seeking a freelance journalist to help evaluate and improve AI using newsroom expertise, editorial judgment, and clear communication. No prior AI experience required. You will design and assess real-world editorial and... 
    Remote job
    Freelance

    Turing

    San Francisco, CA
    4 days ago
  • Ali Aydi is seeking photography professionals to support an AI data quality evaluation project. This remote contract role involves comparing original photographs with AI-generated images to identify inconsistencies and document feedback. Responsibilities include uploading... 
    Remote job
    Contract work

    Ali Aydi

    Tallahassee, FL
    3 days ago
  • $21 - $65 per hour

    EliteBrain is seeking photographers to remotely evaluate and improve visual data quality for AI systems. You will upload original photos, compare them with AI-generated images, and provide detailed feedback in clear English. The role is remote within the United States... 
    Remote job
    Hourly pay
    For contractors

    NEPSE Trading

    Tallahassee, FL
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Conversation Evaluator - Remote. Be the first to apply!