Video Evaluator (AI Content) [Remote]
AuraOne Human Data
- Remote job
Video Evaluator (AI Content) is a remote evaluation track for reviewing video evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can use to retrain.
Why this role matters
AI data reviewers help turn video evaluation outputs into auditable labels, rationales, and regression cases for AuraOne Human Data.
Responsibilities
- Evaluate video evaluation model outputs against a versioned rubric and assign severity tags for Video Evaluator (AI Content) assignments.
- Compare paired responses and pick the stronger answer with a written rationale.
- Label hallucinations, instruction-following failures, and unsafe content with structured tags.
- Capture ambiguous prompts and route them back to the program team for rubric updates.
- Maintain reviewer-quality scores by calibrating against gold-standard examples each week.
- Document recurring failure modes so the modeling team can target them in the next training run.
Qualifications
- Prior evaluation, annotation, or human-rater experience on video evaluation or adjacent content for Video Evaluator (AI Content) work.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that names the issue and the rubric clause being applied.
- Strong attention to detail and the ability to flag when a prompt itself is the problem.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Compare two video evaluation model responses to the same prompt and pick the stronger one with rationale.
- Tag an unsafe response with the correct policy category and severity.
- Audit a 50-row batch for rubric consistency and report drift to the program lead.
- Propose a rubric clarification after spotting a recurring failure mode.
Nice to have
- Background in linguistics, content moderation, or trust & safety review.
- Experience with inter-rater agreement metrics and calibration cycles.
- Domain expertise that lets you spot subject-matter errors automated checks miss.
Skills
- Model output evaluation
- Rubric-based annotation
- Severity tagging
- Inter-rater calibration
- Video evaluation
- Design
- AI model evaluation
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
$30 - $40 per hour
211 Specialist, Government Services AI Evaluator Use your experience helping people find public... ...that answer questions and generate content. Experienced public-service professionals... ...0 to $45 per hour. Review AI-generated videos, compare outputs, and explain which result...ContentVideoHourly payContract workPart timeFor contractorsRemote workFlexible hours- ...Video Understanding AI Evaluator is a remote evaluation track for reviewing video understanding ai evaluation prompts and responses against AuraOne... ...hallucinations, instruction-following failures, and unsafe content with structured tags. Capture ambiguous prompts and...ContentVideoRemote jobHourly payFor contractors10 hours per week
$15 per hour
...and Artificial Intelligence (AI) are inextricably linked. Our... ...intelligence by providing the data, evaluation frameworks, and human... ...Labeling elements of a piece of content rather than the content as a whole... ...), or altering audio/video signals (speed modification, pitch...ContentVideoRemote jobHourly payFull timePart timeFixed term contractFlexible hoursShift work- Lifted is seeking experienced evaluators to assess model-generated content across text, images, audio, video, HTML widgets, PDFs and more. You will apply defined rubrics for factuality and consistency to deliver dependable quality signals for product teams. Applicants...ContentVideoRemote jobFlexible hours
- ...Seeking PhD-level quantitative finance experts, the full-time Remote AI Research Evaluator will assess AI-generated financial content and contribute to the evaluation of advanced AI models in a flexible, remote contract role. Key Responsibilities Assessing the factuality...ContentFull timeContract workRemote workFlexible hours
- YO AI Labs is seeking an experienced Photography and Cinematography Expert to evaluate professional photo and video content for a specialized project. This remote, worldwide role requires expertise in photography, cinematography, camera operation, and video production....ContentVideoRemote jobWorldwide
- ...Seeking an AI-Focused Video Editor for a full-time remote position, responsible for creating and editing AI-assisted video content for paid social advertising on platforms like Meta and TikTok, while collaborating with the creative team to produce high-quality, performance...ContentVideoFull timeRemote work
- Morningside Group is seeking a talented Video Editor & Motion Graphics Producer to oversee post-production processes for YouTube and digital content. You will transform ideas into impactful videos, managing tasks from video editing to motion graphics integration. We're...ContentVideoRemote work
- ...generation and brand awareness for US and UK markets. The role emphasizes B2B content, video, and design leadership, collaborating with sales to drive qualified opportunities. Proficiency with AI tools and modern marketing tech is required. You could be a recent graduate...ContentVideoRemote job
- Alignerr is seeking Wildlife and Habitat Conservation Scientists to evaluate AI-trained biodiversity protection content. This remote, hourly contract role offers flexible hours (10-40 per week) and a chance to shape how AI understands ecological challenges on a global scale...ContentRemote jobHourly payContract workFlexible hours
- agency is seeking a video editing specialist to test and improve an AI model’s understanding of visual storytelling. You’ll evaluate model responses for storytelling, pacing, color, sound, and transitions, documenting errors for reproducibility. Ideal candidates have strong...VideoRemote jobHourly payContract work
- AuraOne, Inc. is seeking an AI Data Reviewer for the Misinformation Context AI Evaluator track. This remote contractor role involves reviewing model outputs against... ...compare paired responses, tag misinfo and unsafe content, and calibrate scores weekly to maintain high...ContentRemote jobFor contractors
- Outlier, operated by Scale AI, is seeking detail-oriented English speakers to review AI-generated media from home. You will roleplay live conversations with an AI voice model and rate outputs, then explain issues in writing for continuous model improvement. You can set...ContentRemote jobWeekly payFreelanceFlexible hours
- Outlier is seeking detail-oriented English speakers to review AI-generated media from home on a flexible schedule. You will roleplay... ...in writing what looked or sounded off. You will compare image and video outputs to judge adherence to prompts and quality, then provide precise...VideoRemote jobFlexible hours
- OpenTrain AI, Inc. seeks professionals with energy accounting and utility operations experience to evaluate AI-generated content and create training data. This remote U.S. contract role focuses on utility metering, billing, customer operations, and customer data management...ContentRemote jobContract work
$15 - $25 per hour
...AI Video Editor Create engaging video content using AI-powered video creation and editing tools. The work may include generating visuals, assembling clips, adding transitions, captions, music, and basic motion graphics. Deliver polished, platform-ready video exports...ContentVideoRemote work- prolificacademicltd is offering a remote, flexible-role focused on evaluating and improving AI-generated customer support content. You will assess model replies, create test tickets and dialogue flows, and review FAQs to ensure clarity and accuracy. The position requires...ContentRemote jobFlexible hours
$30 per hour
Prolific Academic Ltd offers remote paid tasks to help improve AI systems by evaluating automated support conversations and reviewing generated content for accuracy and tone. Participants complete a brief skills test before being invited to join. Ideal contributors have...ContentRemote jobHourly pay- LILT seeks detail-oriented human reviewers to support AI training and evaluation. You will work as an independent contractor across tasks like... ...evaluating prompts, reviewing text, and analyzing multimedia content. Your work depends on applying local cultural context,...ContentRemote jobFor contractorsLocal area
- Prolific Academic Ltd seeks qualified evaluators to help assess and improve AI systems by applying computer science expertise to technical questions, research material and generated content. After a short skills assessment, successful applicants can take paid tasks as independent...ContentRemote job
$85 per hour
OpenTrain AI, Inc. is seeking a Process Safety Engineer AI Training Evaluator to review AI-generated content on environmental health and safety, process safety, and energy compliance. This remote, flexible project pays up to $85 per hour and requires strong written communication...ContentRemote jobHourly payFlexible hours- Vettedlance is seeking a meticulous evaluator to rate and rewrite AI model responses in Modern Standard Arabic. You will assess accuracy, fluency and... ...answers, and flag unsafe, biased or mistranslated content. This is a remote, hourly or per-task contract with flexible...ContentRemote jobHourly payContract workPart timeFlexible hours
- ...understanding of local cultural context to support a range of AI training and evaluation projects. You will work across diverse task types,... ...outputs, reviewing and correcting text, analyzing multimedia content, contributing voice recordings, among others. This independent...ContentFor contractorsLocal areaRemote workFlexible hours
- Revelo is seeking a Health & Fitness Specialist in LLM Training to evaluate AI-generated health, wellness, and exercise content for safety, accuracy, and quality. You will analyze prompts and responses to ensure alignment with exercise science and medical standards. This...ContentPart timeFreelanceRemote workFlexible hours
- Anyone AI Inc. is seeking multilingual coding experts to evaluate AI-generated responses on a part-time remote contract lasting up to 3 months. This opportunity... ...AI-generated customer service and technical content for clarity, correctness, and quality, validating...ContentRemote jobContract workPart time
- ...To produce engaging technical videos for healthcare developers and teams, the full-time Video Content Creator will script, film, edit, and publish content while building... ...and scaling systems, including the use of AI tools Proven ability to understand customer needs...ContentVideoFull timeRemote work
- Handshake AI is seeking experienced Political Science PhDs for flexible, hourly contract work to evaluate AI-generated content and provide structured feedback. This ongoing opportunity focuses on politics, governance, and evidence-based reasoning rather than traditional...ContentRemote jobHourly payContract workFlexible hours
- ...is seeking detail-oriented language reviewers to support AI training and evaluation projects. You will work across tasks such as evaluating prompts... ...and AI outputs, reviewing text, and analyzing multimedia content, applying local cultural context. As a 1099 contractor,...ContentRemote jobFor contractorsLocal area
- Volga Partners is seeking experienced Regulatory CMC Submission Writers to evaluate AI-generated pharmaceutical content remotely as independent contractors. You will apply regulatory and technical expertise to assess accuracy, completeness, and regulatory compliance for...ContentRemote jobFor contractors
- Welo Data is seeking Mandarin-speaking experts for a 4-week remote language evaluation project. You will evaluate AI-generated Mandarin content, write and rewrite prompts, and refine responses to improve dataset quality. This is a freelance contract with flexible weekly...ContentRemote jobContract workFreelanceWork from homeFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Video Evaluator (AI Content) [Remote]. Be the first to apply!

