Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Video Understanding AI Evaluator [Remote]

AuraOne Human Data

Remote
  • Remote job

Video Understanding AI Evaluator is a remote evaluation track for reviewing video understanding ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can use to retrain.

Why this role matters

AI data reviewers help turn video understanding ai evaluation outputs into auditable labels, rationales, and regression cases for AuraOne Human Data.

Responsibilities

  • Evaluate video understanding ai evaluation model outputs against a versioned rubric and assign severity tags for Video Understanding AI Evaluator assignments.
  • Compare paired responses and pick the stronger answer with a written rationale.
  • Label hallucinations, instruction-following failures, and unsafe content with structured tags.
  • Capture ambiguous prompts and route them back to the program team for rubric updates.
  • Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
  • Document recurring failure modes so the modeling team can target them in the next training run.

Qualifications

  • Prior evaluation, annotation, or human-rater experience on video understanding ai evaluation or adjacent content for Video Understanding AI Evaluator work.
  • Comfort applying multi-page rubrics consistently across long batches.
  • Clear written reasoning that names the issue and the rubric clause being applied.
  • Strong attention to detail and the ability to flag when a prompt itself is the problem.
  • Reliable async availability for at least 10 hours per week.

Example tasks

  • Compare two video understanding ai evaluation model responses to the same prompt and pick the stronger one with rationale.
  • Tag an unsafe response with the correct policy category and severity.
  • Audit a 50-row batch for rubric consistency and report drift to the program lead.
  • Propose a rubric clarification after spotting a recurring failure mode.

Nice to have

  • Background in linguistics, content moderation, or trust & safety review.
  • Experience with inter-rater agreement metrics and calibration cycles.
  • Domain expertise that lets you spot subject-matter errors automated checks miss.

Skills

  • Model output evaluation
  • Rubric-based annotation
  • Severity tagging
  • Inter-rater calibration
  • Video Understanding AI evaluation
  • Voice, language and multimodal
  • AI evaluation
  • Rubric writing
  • Expert review

Work model

Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.

Compensation

Hourly rate confirmed after the interview process.

Application process

Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.

Vacancy posted 7 days ago
Similar jobs that could be interesting for youBased on the Video Understanding AI Evaluator [Remote] in Remote vacancy
  • agency is seeking a video editing specialist to test and improve an AI model’s understanding of visual storytelling. You’ll evaluate model responses for storytelling, pacing, color, sound, and transitions, documenting errors for reproducibility. Ideal candidates have strong... 
    Video
    Remote job
    Hourly pay
    Contract work

    agency

    New York, NY
    3 days ago
  • Outlier is seeking detail-oriented English speakers to review AI-generated media from home on a flexible schedule. You will roleplay...  ...in writing what looked or sounded off. You will compare image and video outputs to judge adherence to prompts and quality, then provide precise... 
    Video
    Remote job
    Flexible hours

    Outlier

    Boca Raton, FL
    5 days ago
  • AuraOne, Inc. is seeking a diligent remote contractor to evaluate at-work video recorder outputs against a versioned rubric. You will assign severity tags, write rationales, and help identify edge cases for quality control. Responsibilities include comparing responses,... 
    Video
    Remote job
    For contractors

    AuraOne, Inc.

    New York, NY
    3 days ago
  • $30 - $40 per hour

    211 Specialist, Government Services AI Evaluator Use your experience helping people find public benefits and community resources to evaluate...  ...role pays $20 to $45 per hour. Review AI-generated videos, compare outputs, and explain which result is stronger using your... 
    Video
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    1 day ago
  • $15 per hour

     ...and Artificial Intelligence (AI) are inextricably linked. Our...  ...intelligence by providing the data, evaluation frameworks, and human...  ...Generating labels to advance understanding of a concept, trend etc....  ...summarization), or altering audio/video signals (speed modification,... 
    Video
    Remote job
    Hourly pay
    Full time
    Part time
    Fixed term contract
    Flexible hours
    Shift work

    Innodata Inc.

    Kansas
    1 day ago
  •  ...Seeking an AI-Focused Video Editor for a full-time remote position, responsible for creating and editing AI-assisted video content for...  ...CapCut, or similar professional video editing tools Strong understanding of hooks, pacing, and attention retention in video formats... 
    Video
    Full time
    Remote work

    Virtual Vocations Inc

    United States
    4 days ago
  • $14.5 per hour

     ...diverse, and scalable to supercharge their AI models. As a Welocalize brand, Welo Data...  ...! Job Description As a Web Search Evaluator, you will play a key role in improving...  ...English (written and spoken) Strong understanding of pop culture in the US Reliable... 
    Part time
    Currently hiring
    Immediate start
    Remote work
    Work from home
    10 hours per week
    Flexible hours

    Jobs for Humanity

    United States
    5 days ago
  •  ...Recruitment is working with a fast-growing AI technology company behind an AI...  ...seeking to onboard a Mid/Senior AI Cinematic Video Creator to join their team. The successful...  .... Kling, Seedance, Happy Horse) ~ Solid understanding of AI image and video generation pipelines... 
    Video
    Full time
    Contract work
    Remote work
    Worldwide

    David Kennedy Recruitment

    Remote
    5 days ago
  •  ...narrative-driven content and high-performing video across a range of formats from short-form...  ...storytelling. We’re looking for an AI Specialist to lead AI implementation across...  ...workflows, or content operations Strong understanding of post-production pipelines and editing... 
    Video
    Full time
    Remote work

    Shadow Light Studios

    New York, NY
    4 days ago
  •  ...AI Video Editor You already live inside Reels, TikTok, and Shorts. You know what it takes to keep someone watching past three seconds...  ...(agency, DTC, info products, coaching, etc.). ~ Solid understanding of short-form retention, how hooks, pacing, framing, and captions... 
    Video
    Remote work
    Flexible hours

    Native Commerce

    United States
    4 days ago
  •  ...role in the United States. The project runs 15-25 hours per week and focuses on evaluating and improving AI systems on federal student loan topics. Responsibilities include understanding federal loan programs, identifying incorrect guidance, and addressing borrower questions... 
    Remote job
    Hourly pay
    Temporary work
    Part time
    For contractors

    Softechub

    New York, NY
    1 day ago
  • Alignerr is seeking Wildlife and Habitat Conservation Scientists to evaluate AI-trained biodiversity protection content. This remote, hourly...  ...flexible hours (10-40 per week) and a chance to shape how AI understands ecological challenges on a global scale. Ideal candidates... 
    Remote job
    Hourly pay
    Contract work
    Flexible hours

    Alignerr Corp.

    Miami, FL
    3 days ago
  • Prolific is seeking fluent Kannada speakers to act as evaluators for AI training tasks conducted remotely with flexible hours. You will listen...  ..., and rate naturalness and cultural accuracy, helping AI understand Kannada nuances. Tasks are posted individually; pay is task-... 
    Remote job
    Flexible hours

    Prolific

    New York, NY
    1 day ago
  • LILT is seeking detail-oriented human reviewers with a strong understanding of local cultural context to support a range of AI training and evaluation projects. You will work across diverse task types, including evaluating prompts and AI-generated outputs, reviewing and... 
    For contractors
    Local area
    Remote work
    Flexible hours

    LILT

    New York, NY
    18 hours ago
  • $25 - $35 per hour

    Assess how AI-generated versions differ from the original images and document visual...  ...following detailed review standards; understand common failure modes in AI-generated images...  ...graphic design or photo editing, image evaluation, data annotation, image comparison, and... 
    Remote job
    Hourly pay
    Part time
    For contractors
    Flexible hours

    OpenTrain AI

    New York, NY
    3 days ago
  • $20 per hour

    A tech company specializing in AI is hiring a Digital Web Designer. In this remote role, you will evaluate AI-generated designs and provide feedback to enhance the model’s understanding of aesthetics. An ideal candidate will have a strong background in UI/UX design and... 
    Remote job
    Flexible hours

    DataAnnotation

    Columbia, SC
    5 days ago
  •  ...co-creates tools with clients, and runs real research into what AI-native production can actually do. We're not following the playbook...  ..., not as toys, but to see how far they can be pushed. You understand that prompting is a craft, that on-brand and high-quality don't... 
    Video
    Full time
    Live in
    Remote work
    Weekday work

    Weekday

    United States
    2 days ago
  •  ...To produce engaging technical videos for healthcare developers and teams, the full-time Video Content Creator will script...  ...in designing and scaling systems, including the use of AI tools Proven ability to understand customer needs and translate them into compelling... 
    Video
    Full time
    Remote work

    Virtual Vocations Inc

    United States
    5 days ago
  • $75 per hour

    Logistics And Planning Professionals AI Evaluator Evaluate AI-generated responses and write prompts based on logistics, production planning...  ...written examples and reviews to help artificial intelligence understand professional tasks, language, and useful responses. #J-18808... 
    Hourly pay
    Contract work
    Part time
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    5 days ago
  • A technology company is seeking a Front-End Web Designer to train AI models in understanding design principles. You will review AI-generated designs and provide feedback to enhance the aesthetic quality. This is a remote role, allowing you to work on your own schedule.... 
    Remote job

    DataAnnotation

    Virginia, MN
    5 days ago
  • Create challenging prompts and evaluate large language model responses for accuracy, reasoning...  ...operational execution to assess how well AI handles management-related topics. Create...  ...work behind systems that generate and understand text. Evaluators review model responses,... 
    Contract work
    Part time
    For contractors
    Remote work
    Worldwide

    OpenTrain AI, Inc.

    New York, NY
    3 days ago
  • $8 - $65 per hour

    Role overview Evaluate and improve language models’ ability to answer practical questions about video production. The work draws on cinematography...  ...technical questions with AI models. Check responses for...  ...Final Cut Pro is useful. Understanding of studio and field production... 
    Video
    Hourly pay
    Contract work
    For contractors
    Freelance
    Remote work

    agency

    New York, NY
    2 days ago
  • Prolific is seeking fluent Danish speakers to act as AI trainers, evaluating Danish text and speech for naturalness and cultural accuracy. You...  ..., and provide precise feedback to improve model understanding. This is a remote, task-based role with flexible hours and... 
    Remote job
    Hourly pay
    Flexible hours

    Prolific Academic Ltd

    New York, NY
    1 day ago
  • Feedinkoo is seeking a creative and technically sharp video editor to produce engaging short-form content for social media platforms...  ...2 years of editing experience, a strong portfolio, and a deep understanding of retention-driven storytelling. The position requires expert... 
    Video
    Remote work

    Feedinkoo: Remote Jobs

    New York, NY
    2 days ago
  • $145.6k - $249.6k

     ...Job Overview Join a leading AI lab's cutting-edge Generative...  ...for an experienced Commercial Video Editor and Post-Production...  ...improve how advanced AI models understand and execute professional video...  ...-quality reference edits, and evaluate AI-generated video content for... 
    Video
    Hourly pay
    Full time
    Temporary work
    Remote work
    Monday to Friday
    Weekday work

    Weekday AI

    Remote
    5 days ago
  •  ...Content/Marketing Strategist & Short Video Editor (Remote, US-based) Get AI-powered advice on this job and more exclusive features. We’re looking...  ...or helped create viral short‑form videos before — you understand what makes people stop scrolling, watch, and share.... 
    Video
    Full time
    Part time
    Freelance
    Remote work
    Flexible hours

    Edensign

    United States
    4 days ago
  •  ...world-class stock assets, purpose-built AI tools for creatives need to bring their ideas...  ...demos, creative experiments, social videos, tutorials that don't feel like tutorials...  ...it can make creative work better. You understand social, but you're not interested in just... 
    Video
    Remote work
    Home office
    Flexible hours

    Envato

    United States
    5 days ago
  •  ...Video Content Creator Alexandria, Egypt About the Job Job Title: Video Content...  ...and editing tools and be open to exploring AI-powered tools to enhance creativity,...  ...projects from idea through to completion Understanding of video formats, resolutions, and platform... 
    Video
    Full time
    Contract work
    Part time
    Freelance
    Remote work
    Flexible hours

    5 Star Global Recruitment Partners

    United States
    4 days ago
  •  ...Senior AI Video Editor Trafilea is a Consumer Tech Platform for Transformative Brand Growth. We're building the AI Growth Engine...  ...-generated content integrated into finished work. ~ Strong understanding of direct-response and social-first editing techniques, including... 
    Video
    Remote work

    Trafilea

    United States
    2 days ago
  •  ...calibrated to scale, down to the smallest comma. Video Production - We produce short vertical...  ...designed for performance (UGC, VSL, AI content, etc.), adapted to the codes of...  ...social ads codes (TikTok / Meta). Excellent understanding of rhythm, storytelling and retention.... 
    Video
    Freelance
    Remote work
    Flexible hours

    CHASR

    New York, NY
    3 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Video Understanding AI Evaluator [Remote]. Be the first to apply!