Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Video Evaluator (AI Content) [Remote]

AuraOne Human Data

Remote
  • Remote job

Video Evaluator (AI Content) is a remote evaluation track for reviewing video evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can use to retrain.

Why this role matters

AI data reviewers help turn video evaluation outputs into auditable labels, rationales, and regression cases for AuraOne Human Data.

Responsibilities

  • Evaluate video evaluation model outputs against a versioned rubric and assign severity tags for Video Evaluator (AI Content) assignments.
  • Compare paired responses and pick the stronger answer with a written rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with structured tags.
  • Capture ambiguous prompts and route them back to the program team for rubric updates.
  • Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
  • Document recurring failure modes so the modeling team can target them in the next training run.

Qualifications

  • Prior evaluation, annotation, or human-rater experience on video evaluation or adjacent content for Video Evaluator (AI Content) work.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the issue and the rubric clause being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Example tasks

  • Compare two video evaluation model responses to the same prompt and pick the stronger one with rationale.
  • Tag an unsafe response with the correct policy category and severity.
  • Audit a 50-row batch for rubric consistency and report drift to the program lead.
  • Propose a rubric clarification after spotting a recurring failure mode.

Nice to have

  • Background in linguistics, content moderation, or trust & safety review.
  • Experience with inter-rater agreement metrics and calibration cycles.
  • Domain expertise that lets you spot subject-matter errors automated checks miss.

Skills

  • Model output evaluation
  • Rubric-based annotation
  • Severity tagging
  • Inter-rater calibration
  • Video evaluation
  • Design
  • AI model evaluation

Work model

Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

Hourly rate confirmed after the interview process.

Application process

Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.

Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Video Evaluator (AI Content) [Remote] in Remote vacancy
  • $30 - $40 per hour

    211 Specialist, Government Services AI Evaluator Use your experience helping people find public...  ...that answer questions and generate content. Experienced public-service professionals...  ...0 to $45 per hour. Review AI-generated videos, compare outputs, and explain which result... 
    Content
    Video
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    1 day ago
  •  ...Video Understanding AI Evaluator is a remote evaluation track for reviewing video understanding ai evaluation prompts and responses against AuraOne...  ...hallucinations, instruction-following failures, and unsafe content with structured tags. Capture ambiguous prompts and... 
    Content
    Video
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    7 days ago
  • $15 per hour

     ...and Artificial Intelligence (AI) are inextricably linked. Our...  ...intelligence by providing the data, evaluation frameworks, and human...  ...Labeling elements of a piece of content rather than the content as a whole...  ...), or altering audio/video signals (speed modification, pitch... 
    Content
    Video
    Remote job
    Hourly pay
    Full time
    Part time
    Fixed term contract
    Flexible hours
    Shift work

    Innodata Inc.

    Kansas
    1 day ago
  • Lifted is seeking experienced evaluators to assess model-generated content across text, images, audio, video, HTML widgets, PDFs and more. You will apply defined rubrics for factuality and consistency to deliver dependable quality signals for product teams. Applicants... 
    Content
    Video
    Remote job
    Flexible hours

    Lifted (Upwork)

    New York, NY
    5 days ago
  •  ...Seeking PhD-level quantitative finance experts, the full-time Remote AI Research Evaluator will assess AI-generated financial content and contribute to the evaluation of advanced AI models in a flexible, remote contract role. Key Responsibilities Assessing the factuality... 
    Content
    Full time
    Contract work
    Remote work
    Flexible hours

    Virtual Vocations Inc

    United States
    3 days ago
  • YO AI Labs is seeking an experienced Photography and Cinematography Expert to evaluate professional photo and video content for a specialized project. This remote, worldwide role requires expertise in photography, cinematography, camera operation, and video production.... 
    Content
    Video
    Remote job
    Worldwide

    YO AI Labs

    New York, NY
    5 days ago
  •  ...Seeking an AI-Focused Video Editor for a full-time remote position, responsible for creating and editing AI-assisted video content for paid social advertising on platforms like Meta and TikTok, while collaborating with the creative team to produce high-quality, performance... 
    Content
    Video
    Full time
    Remote work

    Virtual Vocations Inc

    United States
    4 days ago
  • Morningside Group is seeking a talented Video Editor & Motion Graphics Producer to oversee post-production processes for YouTube and digital content. You will transform ideas into impactful videos, managing tasks from video editing to motion graphics integration. We're... 
    Content
    Video
    Remote work

    Feedinkoo: Remote Jobs

    New York, NY
    3 days ago
  •  ...generation and brand awareness for US and UK markets. The role emphasizes B2B content, video, and design leadership, collaborating with sales to drive qualified opportunities. Proficiency with AI tools and modern marketing tech is required. You could be a recent graduate... 
    Content
    Video
    Remote job

    Revelwood Incorporated

    Florham Park, NJ
    3 days ago
  • Alignerr is seeking Wildlife and Habitat Conservation Scientists to evaluate AI-trained biodiversity protection content. This remote, hourly contract role offers flexible hours (10-40 per week) and a chance to shape how AI understands ecological challenges on a global scale... 
    Content
    Remote job
    Hourly pay
    Contract work
    Flexible hours

    Alignerr Corp.

    Miami, FL
    3 days ago
  • agency is seeking a video editing specialist to test and improve an AI model’s understanding of visual storytelling. You’ll evaluate model responses for storytelling, pacing, color, sound, and transitions, documenting errors for reproducibility. Ideal candidates have strong... 
    Video
    Remote job
    Hourly pay
    Contract work

    agency

    New York, NY
    3 days ago
  • AuraOne, Inc. is seeking an AI Data Reviewer for the Misinformation Context AI Evaluator track. This remote contractor role involves reviewing model outputs against...  ...compare paired responses, tag misinfo and unsafe content, and calibrate scores weekly to maintain high... 
    Content
    Remote job
    For contractors

    AuraOne, Inc.

    New York, NY
    5 days ago
  • Outlier, operated by Scale AI, is seeking detail-oriented English speakers to review AI-generated media from home. You will roleplay live conversations with an AI voice model and rate outputs, then explain issues in writing for continuous model improvement. You can set... 
    Content
    Remote job
    Weekly pay
    Freelance
    Flexible hours

    Outlier (by Scale AI)

    New York, NY
    3 days ago
  • Outlier is seeking detail-oriented English speakers to review AI-generated media from home on a flexible schedule. You will roleplay...  ...in writing what looked or sounded off. You will compare image and video outputs to judge adherence to prompts and quality, then provide precise... 
    Video
    Remote job
    Flexible hours

    Outlier

    Boca Raton, FL
    5 days ago
  • OpenTrain AI, Inc. seeks professionals with energy accounting and utility operations experience to evaluate AI-generated content and create training data. This remote U.S. contract role focuses on utility metering, billing, customer operations, and customer data management... 
    Content
    Remote job
    Contract work

    OpenTrain AI, Inc.

    New York, NY
    5 days ago
  • $15 - $25 per hour

     ...AI Video Editor Create engaging video content using AI-powered video creation and editing tools. The work may include generating visuals, assembling clips, adding transitions, captions, music, and basic motion graphics. Deliver polished, platform-ready video exports... 
    Content
    Video
    Remote work

    Twine

    United States
    5 days ago
  • prolificacademicltd is offering a remote, flexible-role focused on evaluating and improving AI-generated customer support content. You will assess model replies, create test tickets and dialogue flows, and review FAQs to ensure clarity and accuracy. The position requires... 
    Content
    Remote job
    Flexible hours

    prolificacademicltd

    New York, NY
    2 days ago
  • $30 per hour

    Prolific Academic Ltd offers remote paid tasks to help improve AI systems by evaluating automated support conversations and reviewing generated content for accuracy and tone. Participants complete a brief skills test before being invited to join. Ideal contributors have... 
    Content
    Remote job
    Hourly pay

    prolificacademicltd

    New York, NY
    3 days ago
  • LILT seeks detail-oriented human reviewers to support AI training and evaluation. You will work as an independent contractor across tasks like...  ...evaluating prompts, reviewing text, and analyzing multimedia content. Your work depends on applying local cultural context,... 
    Content
    Remote job
    For contractors
    Local area

    LILT

    New York, NY
    1 day ago
  • Prolific Academic Ltd seeks qualified evaluators to help assess and improve AI systems by applying computer science expertise to technical questions, research material and generated content. After a short skills assessment, successful applicants can take paid tasks as independent... 
    Content
    Remote job

    prolificacademicltd

    New York, NY
    3 days ago
  • $85 per hour

    OpenTrain AI, Inc. is seeking a Process Safety Engineer AI Training Evaluator to review AI-generated content on environmental health and safety, process safety, and energy compliance. This remote, flexible project pays up to $85 per hour and requires strong written communication... 
    Content
    Remote job
    Hourly pay
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    3 days ago
  • Vettedlance is seeking a meticulous evaluator to rate and rewrite AI model responses in Modern Standard Arabic. You will assess accuracy, fluency and...  ...answers, and flag unsafe, biased or mistranslated content. This is a remote, hourly or per-task contract with flexible... 
    Content
    Remote job
    Hourly pay
    Contract work
    Part time
    Flexible hours

    Vettedlance

    New York, NY
    18 hours ago
  •  ...understanding of local cultural context to support a range of AI training and evaluation projects. You will work across diverse task types,...  ...outputs, reviewing and correcting text, analyzing multimedia content, contributing voice recordings, among others. This independent... 
    Content
    For contractors
    Local area
    Remote work
    Flexible hours

    LILT

    New York, NY
    18 hours ago
  • Revelo is seeking a Health & Fitness Specialist in LLM Training to evaluate AI-generated health, wellness, and exercise content for safety, accuracy, and quality. You will analyze prompts and responses to ensure alignment with exercise science and medical standards. This... 
    Content
    Part time
    Freelance
    Remote work
    Flexible hours

    Revelo

    New York, NY
    5 days ago
  • Anyone AI Inc. is seeking multilingual coding experts to evaluate AI-generated responses on a part-time remote contract lasting up to 3 months. This opportunity...  ...AI-generated customer service and technical content for clarity, correctness, and quality, validating... 
    Content
    Remote job
    Contract work
    Part time

    Anyone AI Inc.

    New York, NY
    3 days ago
  •  ...To produce engaging technical videos for healthcare developers and teams, the full-time Video Content Creator will script, film, edit, and publish content while building...  ...and scaling systems, including the use of AI tools Proven ability to understand customer needs... 
    Content
    Video
    Full time
    Remote work

    Virtual Vocations Inc

    United States
    5 days ago
  • Handshake AI is seeking experienced Political Science PhDs for flexible, hourly contract work to evaluate AI-generated content and provide structured feedback. This ongoing opportunity focuses on politics, governance, and evidence-based reasoning rather than traditional... 
    Content
    Remote job
    Hourly pay
    Contract work
    Flexible hours

    Handshake

    New York, NY
    5 days ago
  •  ...is seeking detail-oriented language reviewers to support AI training and evaluation projects. You will work across tasks such as evaluating prompts...  ...and AI outputs, reviewing text, and analyzing multimedia content, applying local cultural context. As a 1099 contractor,... 
    Content
    Remote job
    For contractors
    Local area

    LILT

    New York, NY
    1 day ago
  • Volga Partners is seeking experienced Regulatory CMC Submission Writers to evaluate AI-generated pharmaceutical content remotely as independent contractors. You will apply regulatory and technical expertise to assess accuracy, completeness, and regulatory compliance for... 
    Content
    Remote job
    For contractors

    Volga Partners

    New York, NY
    2 days ago
  • Welo Data is seeking Mandarin-speaking experts for a 4-week remote language evaluation project. You will evaluate AI-generated Mandarin content, write and rewrite prompts, and refine responses to improve dataset quality. This is a freelance contract with flexible weekly... 
    Content
    Remote job
    Contract work
    Freelance
    Work from home
    Flexible hours

    Welo Global

    New York, NY
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Video Evaluator (AI Content) [Remote]. Be the first to apply!