Tutor Conversation AI Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Tutor Conversation AI Evaluator is a remote review track for evaluating AI outputs across tutor conversation ai specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it.
Why this role matters
Tutor Conversation AI specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done.
Responsibilities
- Review AI outputs against current tutor conversation ai specialist operations workflows, playbooks, and firm policy for Tutor Conversation AI Evaluator assignments.
- Grade tone, escalation logic, and stakeholder fit on a structured rubric.
- Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
- Capture the right next step so the modeling team can train on it.
- Adjudicate disputed treatments against published playbooks or firm guidance.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Direct working experience in tutor conversation ai specialist operations on real teams for Tutor Conversation AI Evaluator work.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that names the policy or workflow being applied.
- Strong attention to detail and the ability to flag when a prompt itself is the problem.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Grade a model's response to a real tutor conversation ai specialist operations ticket and rate workflow, tone, and escalation.
- Flag a missed escalation with the right severity tag and corrected next step.
- Adjudicate a disputed playbook call between two reviewers using firm guidance.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Prior experience training, calibrating, or QA-ing operations teams.
- Familiarity with AI-assisted workflow tooling and its failure modes.
- Bilingual experience for cross-region operations.
Skills
- Operational review
- Policy adherence
- Workflow judgment
- Stakeholder communication
- Tutor Conversation AI specialist operations
- Learning design
- Assessment review
- Pedagogy
- Tutor
- Conversation
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Tutor Conversation AI Evaluator [Remote] in Remote vacancy
- German Voice AI Evaluator (Healthcare Front Desk Conversations) What if your native German fluency could help teach AI how to handle a real phone call to a doctor's office? We're looking for native German speakers based in the New York area to hold natural spoken conversations...SuggestedContract workWork at officeRemote workFlexible hours
- Lifted is seeking experts to engage in live conversations with AI systems, review and label transcripts, and provide written justifications based on rubrics. You will assess language quality, technical accuracy, and JSON formatting while capturing session artifacts and...SuggestedRemote workFlexible hours
- Anyone AI is seeking experts to evaluate personalized AI conversations by engaging in natural, back-and-forth dialogues with two AI assistants. You will compare how well each assistant uses information from your connected services and prior activity to address your questions...SuggestedRemote jobPart time
- ...About the role AI assistants are becoming part of everyday life, helping people... ...own ChatGPT account, engage in everyday conversations, and share your feedback on what worked,... ...and select your preferred experience. Evaluate conversational quality, helpfulness, clarity...SuggestedHourly payTemporary workPart timeRemote work
micro1
San Francisco, CA2 days ago $20 - $160 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Generalist Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $20–$160/hour...SuggestedContract workSummer workRemote work- ...Conversational AI Evaluator is a remote evaluation track for reviewing conversational ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling team can...Remote jobHourly payFor contractors10 hours per week
- ...An innovative education provider seeks a freelance teacher to guide students in Conversation Techniques. In this flexible role, you will craft engaging course materials and support students through their learning journey. With the freedom to shape your curriculum and...FreelanceRemote workFlexible hours
Coursle.com, part of Ondernemersschool BV
New York, NY1 day ago $72.8k - $93.6k
...from people using those products shapes what we build next. AI TUTOR, ESTONIAN - TEMPORARY In this role, you will help train and... ...dictate LENGTH OF POSITION: 6 months (extensions or conversion to a permanent role may be available based on business needs)...Permanent employmentFull timeTemporary workPart timeLocal areaRemote work- Outlier is seeking detail-oriented English speakers to review AI-generated media from home on a flexible schedule. You will roleplay live conversations with AI voice models, score exchanges, and explain in writing what looked or sounded off. You will compare image and video...Remote jobFlexible hours
- Outlier is seeking detail-oriented English speakers to review AI-generated media from home with flexible scheduling. You will roleplay with AI voice models, rate conversations, and compare visuals to assess adherence to prompts, then document findings in English. Ideal...Remote jobFlexible hours
- ...for a remote contractor role focused on an internal AI training project. You will bring your sales experience... ...through written and verbal communication. You will evaluate AI interview scenarios, simulate conversations across industries, document observations, and share...Remote jobFor contractors
- Outlier, operated by Scale AI, is seeking detail-oriented English speakers to review AI-generated media from home. You will roleplay live conversations with an AI voice model and rate outputs, then explain issues in writing for continuous model improvement. You can set...Remote jobWeekly payFreelanceFlexible hours
- Role Title: Food Delivery AI Response Evaluator Role Type : Contractor Location: Remote our partner is engaging Food Delivery AI Response... ...suggestions for improvement. Create illustrative example conversations that reflect authentic food delivery customer interactions...Contract workFor contractorsRemote workWorldwideShift work
- Jobgether is an AI-powered platform matching candidates with hiring companies. This freelance trainer project focuses on evaluating conversational AI through realistic audio interactions conducted remotely, offering flexible hours and independent contractor status. You...Remote jobFor contractorsFreelanceFlexible hours
- Zekai is seeking a Food Delivery AI Response Evaluator to work remotely as a contractor. You will assess AI-generated replies in food delivery... ...and customer service helps. You will draft example conversations and provide concise improvement suggestions, collaborating...Remote jobFor contractors
$18 per hour
Outlier is seeking detail-oriented English speakers to review AI-generated media. You will hold live conversations with an AI voice model, score exchanges for naturalness and usefulness, and explain in writing what looked or sounded off, all remotely and on your own schedule...Remote jobFlexible hours$30 per hour
Business Analyst, Educational Rater and Evaluator Review and improve AI-generated educational responses through research, fact checking, and clear... ...with genuine prior usage and at least 10 relevant conversations about learning, education, research, projects, exam preparation...Hourly payContract workPart timeFor contractorsRemote work$40 per hour
Talantir is seeking a remote contractor to work on evaluating AI chatbots. You will be given coding challenges for AI chatbots and assess... ...and set your own schedule. PayPal handles payments; currency conversions are managed by PayPal. #J-18808-Ljbffr DataAnnotationRemote jobHourly payContract workFor contractors- Alignerr is seeking native German speakers in the New York area to evaluate a healthcare AI receptionist by role-playing as a patient and recording conversations. You will assess how well the AI listens, responds, and takes action, then provide structured feedback. This...Remote jobContract workFlexible hours
- Alignerr is seeking native Dutch speakers for a remote contract role to evaluate AI-powered healthcare receptionist interactions. You will role-play as a patient, record short conversations, and assess how well the AI listens, responds, and takes action. You will review...Remote jobContract workFlexible hours
- Dutch Voice AI Evaluator (Healthcare Front Desk Conversations) About the Role What if your voice could help teach AI how to handle a real phone call to a doctor's office? We're looking for native Dutch speakers to hold natural spoken conversations with an AI healthcare...Contract workWork at officeRemote workFlexible hours
$40 - $80 per hour
...Job Description Job Description Job Description: Paralegal – AI Training & Legal Evaluation Role Type: Contractor Location: Remote (Worldwide, U.S. Practice Experience Required) Weekly Commitment: Approximately 10+ Hours About the Role We are seeking...Remote jobHourly payFor contractorsWorldwide10 hours per weekFlexible hours$45 - $70 per hour
...Regular ChatGPT users chat with different versions of an AI assistant through their own account, compare the experiences and submit... ...communication Preferred: English proficiency No coding, AI evaluation experience or specific professional background required Suits...Hourly payContract workRemote workWorldwide$35 - $75 per hour
...using those products shapes what we build next. AI TUTOR, WRITING - TEMPORARY In this role, you will evaluate, refine, and create expert-level writing across... ...LENGTH OF POSITION: 6 months (extensions or conversion to a permanent role may be available based on business...Permanent employmentFull timeTemporary workPart timeLocal areaRemote workShift workxAI
Remote5 days ago- HELP SHAPE THE FUTURE OF AI BY EVALUATING HOW WELL AI ASSISTANTS UNDERSTAND USERS, USE PERSONAL CONTEXT, AND DELIVER HELPFUL CONVERSATIONS. Location: Remote Type: Independent contractor / Part-time Project duration: up to 3 months ABOUT THE ROLE At Anyone AI, we create...Part timeFor contractorsRemote work
$40 - $75 per hour
..., motion graphics, and visual effects to evaluate and annotate video data that reflects how... ...matchmoving, CG compositing). Familiarity with AI video generation tools (Grok Imagine,... ...contracts of 6 months. Extensions or conversion to a permanent role may be available based...Permanent employmentFull timeContract workTemporary workPart timeFixed term contractFreelanceRemote workShift workWeekend work$35 - $75 per hour
...next. ABOUT THE ROLE You will evaluate, refine, and create elite-... ...to label, annotate, and refine AI-generated text for clarity, structure... ...impacts (e.g., 30%+ conversion boosts, engagement/revenue growth... ...LOCATION & OTHER EXPECTATIONS Tutor roles may be offered as full‑time...Full timeTemporary workPart timeFor contractorsRemote workWorldwide10 hours per week$35 - $75 per hour
...those products shapes what we build next. AI TUTOR, HUMANITIES - TEMPORARY In this role,... ...arts to support its training. You will evaluate model outputs and create training data so... ...LENGTH OF POSITION: 6 months (extensions or conversion to a permanent role may be available...Permanent employmentFull timeTemporary workPart timeLocal areaRemote work- Outlier is seeking detail-oriented English speakers to review AI-generated media. You will roleplay live conversations with an AI voice model, score exchanges, and explain in writing what looked or sounded off. This remote role offers flexible hours and allows you to work...Remote jobWork from homeFlexible hours
$18 per hour
Get paid to spot what AI-generated media gets wrong Hold live conversations with AI voice models, compare image and video pairs, and explain what’s off. Fully... ...reason, learn, and improve. Through Outlier’s evaluation and feedback loop, your work helps turn frontier research...Remote jobWeekly payFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Tutor Conversation AI Evaluator [Remote]. Be the first to apply!
