Video Understanding AI Evaluator [Remote]
AuraOne Human Data
- Remote job
Video Understanding AI Evaluator is a remote evaluation track for reviewing video understanding ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can use to retrain.
Why this role matters
AI data reviewers help turn video understanding ai evaluation outputs into auditable labels, rationales, and regression cases for AuraOne Human Data.
Responsibilities
- Evaluate video understanding ai evaluation model outputs against a versioned rubric and assign severity tags for Video Understanding AI Evaluator assignments.
- Compare paired responses and pick the stronger answer with a written rationale.
- Label hallucinations, instruction-following failures, and unsafe content with structured tags.
- Capture ambiguous prompts and route them back to the program team for rubric updates.
- Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
- Document recurring failure modes so the modeling team can target them in the next training run.
Qualifications
- Prior evaluation, annotation, or human-rater experience on video understanding ai evaluation or adjacent content for Video Understanding AI Evaluator work.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that names the issue and the rubric clause being applied.
- Strong attention to detail and the ability to flag when a prompt itself is the problem.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Compare two video understanding ai evaluation model responses to the same prompt and pick the stronger one with rationale.
- Tag an unsafe response with the correct policy category and severity.
- Audit a 50-row batch for rubric consistency and report drift to the program lead.
- Propose a rubric clarification after spotting a recurring failure mode.
Nice to have
- Background in linguistics, content moderation, or trust & safety review.
- Experience with inter-rater agreement metrics and calibration cycles.
- Domain expertise that lets you spot subject-matter errors automated checks miss.
Skills
- Model output evaluation
- Rubric-based annotation
- Severity tagging
- Inter-rater calibration
- Video Understanding AI evaluation
- Voice, language and multimodal
- AI evaluation
- Rubric writing
- Expert review
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
- agency is seeking a video editing specialist to test and improve an AI model’s understanding of visual storytelling. You’ll evaluate model responses for storytelling, pacing, color, sound, and transitions, documenting errors for reproducibility. Ideal candidates have strong...VideoRemote jobHourly payContract work
- Outlier is seeking detail-oriented English speakers to review AI-generated media from home on a flexible schedule. You will roleplay... ...in writing what looked or sounded off. You will compare image and video outputs to judge adherence to prompts and quality, then provide precise...VideoRemote jobFlexible hours
- AuraOne, Inc. is seeking a diligent remote contractor to evaluate at-work video recorder outputs against a versioned rubric. You will assign severity tags, write rationales, and help identify edge cases for quality control. Responsibilities include comparing responses,...VideoRemote jobFor contractors
$30 - $40 per hour
211 Specialist, Government Services AI Evaluator Use your experience helping people find public benefits and community resources to evaluate... ...role pays $20 to $45 per hour. Review AI-generated videos, compare outputs, and explain which result is stronger using your...VideoHourly payContract workPart timeFor contractorsRemote workFlexible hours$15 per hour
...and Artificial Intelligence (AI) are inextricably linked. Our... ...intelligence by providing the data, evaluation frameworks, and human... ...Generating labels to advance understanding of a concept, trend etc.... ...summarization), or altering audio/video signals (speed modification,...VideoRemote jobHourly payFull timePart timeFixed term contractFlexible hoursShift work- ...Seeking an AI-Focused Video Editor for a full-time remote position, responsible for creating and editing AI-assisted video content for... ...CapCut, or similar professional video editing tools Strong understanding of hooks, pacing, and attention retention in video formats...VideoFull timeRemote work
$14.5 per hour
...diverse, and scalable to supercharge their AI models. As a Welocalize brand, Welo Data... ...! Job Description As a Web Search Evaluator, you will play a key role in improving... ...English (written and spoken) Strong understanding of pop culture in the US Reliable...Part timeCurrently hiringImmediate startRemote workWork from home10 hours per weekFlexible hours- ...Recruitment is working with a fast-growing AI technology company behind an AI... ...seeking to onboard a Mid/Senior AI Cinematic Video Creator to join their team. The successful... .... Kling, Seedance, Happy Horse) ~ Solid understanding of AI image and video generation pipelines...VideoFull timeContract workRemote workWorldwide
- ...narrative-driven content and high-performing video across a range of formats from short-form... ...storytelling. We’re looking for an AI Specialist to lead AI implementation across... ...workflows, or content operations Strong understanding of post-production pipelines and editing...VideoFull timeRemote work
- ...AI Video Editor You already live inside Reels, TikTok, and Shorts. You know what it takes to keep someone watching past three seconds... ...(agency, DTC, info products, coaching, etc.). ~ Solid understanding of short-form retention, how hooks, pacing, framing, and captions...VideoRemote workFlexible hours
- ...role in the United States. The project runs 15-25 hours per week and focuses on evaluating and improving AI systems on federal student loan topics. Responsibilities include understanding federal loan programs, identifying incorrect guidance, and addressing borrower questions...Remote jobHourly payTemporary workPart timeFor contractors
- Alignerr is seeking Wildlife and Habitat Conservation Scientists to evaluate AI-trained biodiversity protection content. This remote, hourly... ...flexible hours (10-40 per week) and a chance to shape how AI understands ecological challenges on a global scale. Ideal candidates...Remote jobHourly payContract workFlexible hours
- Prolific is seeking fluent Kannada speakers to act as evaluators for AI training tasks conducted remotely with flexible hours. You will listen... ..., and rate naturalness and cultural accuracy, helping AI understand Kannada nuances. Tasks are posted individually; pay is task-...Remote jobFlexible hours
- LILT is seeking detail-oriented human reviewers with a strong understanding of local cultural context to support a range of AI training and evaluation projects. You will work across diverse task types, including evaluating prompts and AI-generated outputs, reviewing and...For contractorsLocal areaRemote workFlexible hours
$25 - $35 per hour
Assess how AI-generated versions differ from the original images and document visual... ...following detailed review standards; understand common failure modes in AI-generated images... ...graphic design or photo editing, image evaluation, data annotation, image comparison, and...Remote jobHourly payPart timeFor contractorsFlexible hours$20 per hour
A tech company specializing in AI is hiring a Digital Web Designer. In this remote role, you will evaluate AI-generated designs and provide feedback to enhance the model’s understanding of aesthetics. An ideal candidate will have a strong background in UI/UX design and...Remote jobFlexible hours- ...co-creates tools with clients, and runs real research into what AI-native production can actually do. We're not following the playbook... ..., not as toys, but to see how far they can be pushed. You understand that prompting is a craft, that on-brand and high-quality don't...VideoFull timeLive inRemote workWeekday work
- ...To produce engaging technical videos for healthcare developers and teams, the full-time Video Content Creator will script... ...in designing and scaling systems, including the use of AI tools Proven ability to understand customer needs and translate them into compelling...VideoFull timeRemote work
$75 per hour
Logistics And Planning Professionals AI Evaluator Evaluate AI-generated responses and write prompts based on logistics, production planning... ...written examples and reviews to help artificial intelligence understand professional tasks, language, and useful responses. #J-18808...Hourly payContract workPart timeRemote workFlexible hours- A technology company is seeking a Front-End Web Designer to train AI models in understanding design principles. You will review AI-generated designs and provide feedback to enhance the aesthetic quality. This is a remote role, allowing you to work on your own schedule....Remote job
- Create challenging prompts and evaluate large language model responses for accuracy, reasoning... ...operational execution to assess how well AI handles management-related topics. Create... ...work behind systems that generate and understand text. Evaluators review model responses,...Contract workPart timeFor contractorsRemote workWorldwide
$8 - $65 per hour
Role overview Evaluate and improve language models’ ability to answer practical questions about video production. The work draws on cinematography... ...technical questions with AI models. Check responses for... ...Final Cut Pro is useful. Understanding of studio and field production...VideoHourly payContract workFor contractorsFreelanceRemote work- Prolific is seeking fluent Danish speakers to act as AI trainers, evaluating Danish text and speech for naturalness and cultural accuracy. You... ..., and provide precise feedback to improve model understanding. This is a remote, task-based role with flexible hours and...Remote jobHourly payFlexible hours
- Feedinkoo is seeking a creative and technically sharp video editor to produce engaging short-form content for social media platforms... ...2 years of editing experience, a strong portfolio, and a deep understanding of retention-driven storytelling. The position requires expert...VideoRemote work
$145.6k - $249.6k
...Job Overview Join a leading AI lab's cutting-edge Generative... ...for an experienced Commercial Video Editor and Post-Production... ...improve how advanced AI models understand and execute professional video... ...-quality reference edits, and evaluate AI-generated video content for...VideoHourly payFull timeTemporary workRemote workMonday to FridayWeekday work- ...Content/Marketing Strategist & Short Video Editor (Remote, US-based) Get AI-powered advice on this job and more exclusive features. We’re looking... ...or helped create viral short‑form videos before — you understand what makes people stop scrolling, watch, and share....VideoFull timePart timeFreelanceRemote workFlexible hours
- ...world-class stock assets, purpose-built AI tools for creatives need to bring their ideas... ...demos, creative experiments, social videos, tutorials that don't feel like tutorials... ...it can make creative work better. You understand social, but you're not interested in just...VideoRemote workHome officeFlexible hours
- ...Video Content Creator Alexandria, Egypt About the Job Job Title: Video Content... ...and editing tools and be open to exploring AI-powered tools to enhance creativity,... ...projects from idea through to completion Understanding of video formats, resolutions, and platform...VideoFull timeContract workPart timeFreelanceRemote workFlexible hours
- ...Senior AI Video Editor Trafilea is a Consumer Tech Platform for Transformative Brand Growth. We're building the AI Growth Engine... ...-generated content integrated into finished work. ~ Strong understanding of direct-response and social-first editing techniques, including...VideoRemote work
- ...calibrated to scale, down to the smallest comma. Video Production - We produce short vertical... ...designed for performance (UGC, VSL, AI content, etc.), adapted to the codes of... ...social ads codes (TikTok / Meta). Excellent understanding of rhythm, storytelling and retention....VideoFreelanceRemote workFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Video Understanding AI Evaluator [Remote]. Be the first to apply!

