Tutor Conversation AI Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Tutor Conversation AI Evaluator is a remote review track for evaluating AI outputs across tutor conversation ai specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it.
Why this role matters
Tutor Conversation AI specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done.
Responsibilities
- Review AI outputs against current tutor conversation ai specialist operations workflows, playbooks, and firm policy for Tutor Conversation AI Evaluator assignments.
- Grade tone, escalation logic, and stakeholder fit on a structured rubric.
- Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
- Capture the right next step so the modeling team can train on it.
- Adjudicate disputed treatments against published playbooks or firm guidance.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Direct working experience in tutor conversation ai specialist operations on real teams for Tutor Conversation AI Evaluator work.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that names the policy or workflow being applied.
- Strong attention to detail and the ability to flag when a prompt itself is the problem.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Grade a model's response to a real tutor conversation ai specialist operations ticket and rate workflow, tone, and escalation.
- Flag a missed escalation with the right severity tag and corrected next step.
- Adjudicate a disputed playbook call between two reviewers using firm guidance.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Prior experience training, calibrating, or QA-ing operations teams.
- Familiarity with AI-assisted workflow tooling and its failure modes.
- Bilingual experience for cross-region operations.
Skills
- Operational review
- Policy adherence
- Workflow judgment
- Stakeholder communication
- Tutor Conversation AI specialist operations
- Learning design
- Assessment review
- Pedagogy
- Tutor
- Conversation
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 19 days ago
Similar jobs that could be interesting for youBased on the Tutor Conversation AI Evaluator [Remote] in Remote vacancy
- ...counseling, communication, qualitative research, HCI, conflict resolution, or related advisory disciplines to support an AI conversation-evaluation project. The role involves reviewing conversations between people and AI systems, assessing whether the AI gathered...SuggestedContract work
Gramian Consulting
Remotea month ago $14.5 per hour
...AI Web Search Evaluator Unlock the Power of the Internet! Are you curious, tech-savvy, and passionate about improving online search experiences? Join our team as a Web Search Evaluator and help shape the future of search engines from the comfort of your home! As...SuggestedBi-weekly payHourly payPart timeImmediate startRemote workWork from homeFlexible hoursWelo Data
United States1 day ago- ...BAM Ventures is seeking Swedish-speaking remote annotators to evaluate AI-generated content, ensuring that it's coherent and aligns with real-world expectations. Your role will involve reviewing outputs, identifying deviations, and providing structured feedback to enhance...SuggestedRemote work
BAM VENTURES LLC
New York, NY1 day ago $11.5 per hour
...position as an Online Task Contributor. In this role, you will evaluate and provide feedback on content to enhance search engine results... ...11.50 hourly, based on task completion, with a supportive community of contributors involved in AI advancements. #J-18808-Ljbffr...SuggestedHourly payPart timeRemote workUniversity of Delaware
United States3 days ago- ...MERIT Beauty is seeking Turkish-speaking annotators for a contract role evaluating AI-generated content. You will review content for coherence, consistency, and alignment with real-world expectations in Turkish. The ideal candidates are native or fluent Turkish speakers...SuggestedContract workTemporary workImmediate startRemote work
MERIT Beauty
New York, NY3 days ago $35 - $45 per hour
...AI Tutor - Hebrew Remote United States SpaceXAI's mission is to create AI systems that can accurately understand the universe... ...Demonstrated ability to handle multilingual audio content, including evaluating speech accuracy, cultural vocal expressions, and contextual...Full timePart timeFor contractorsRemote workWorldwide10 hours per weekXai
United States4 days ago- Alignerr is seeking a Population Health Informaticist to help train and evaluate health AI on large-scale datasets and public health strategy. You will review AI-generated health insights, assess data-driven metrics, and provide structured feedback that reflects disparities...Remote jobHourly payContract workFlexible hours
$20 - $30 per hour
A remote-focused technology firm is looking for an individual proficient in evaluating large language model outputs. The role involves assessing AI systems, reviewing workflows, and providing actionable feedback to enhance product quality. Ideal candidates will have strong...Remote jobHourly pay- YO AI Labs is seeking an Investment & Finance Expert to work remotely as a contractor. The role focuses on evaluating AI outputs related to valuation, financial modeling, markets, and investments, and crafting expert prompts plus reference answers to improve model reasoning...Remote jobFor contractors
- Feedinkoo is seeking a Graphic Designer to help train and improve AI models focused on UI/UX design, visuals, and user experiences. You will critique AI outputs, provide feedback, and guide model improvements to better reflect aesthetics and usability for designers. The...Remote work
- Dorado is seeking an AI Language Quality Evaluator fluent in Greek and English for an ongoing, task-based project. This remote freelance role involves reviewing translated and AI-flagged content to judge accuracy, classify issues, and suggest corrected translations. You...Remote jobFor contractorsFreelanceFlexible hours
$40 - $100 per hour
About OpenTrain OpenTrain AI is the hiring and contracting organization for this opportunity. OpenTrain is the #1 platform for finding... ...English-language work About AI Training and Scientific Evaluation AI training is the human side of building modern artificial intelligence...Hourly payContract workPart timeFor contractorsRemote work$14.5 per hour
Join to apply for the AI Web Search Evaluator role at Welo Data Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant, diverse, and scalable to supercharge their AI models. As a Welocalize brand, Welo Data leverages...Hourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hours$14.5 per hour
A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical...Remote jobHourly payPart timeFlexible hours- Obsidian is hiring expert Evaluators in real estate, hospitality, and events to review AI-generated work products for accuracy and quality. This is a remote position requiring strong expertise in the relevant domains. Ideal candidates should have over 5 years of professional...Remote jobWork at office
- A leading AI research accelerator is hiring a position focused on contributing to projects that evaluate and enhance AI systems. You will design community service scenarios, write structured explanations, and evaluate AI accuracy. The ideal candidate will have 4+ years...Remote jobFull timeFor contractors
- Obsidian is hiring expert Evaluators in Healthcare operations for a remote, hourly engagement. You will review AI-generated work products for accuracy, rigor, and domain quality, leveraging your extensive subject-matter expertise in the field. The ideal candidate has over...Remote jobHourly payWork at office
$20 - $26 per hour
Prolific is seeking fluent Kannada speakers to act as evaluators. You will assess how naturally and authentically AI captures Kannada speech, by listening to audio clips and comparing text and voice. This fast-paced project pays $20-26 per hour and may require about one...Remote jobHourly payWork from homeFlexible hours$35 - $45 per hour
...A forward-thinking tech company is seeking an AI Tutor specializing in multilingual audio capabilities. This role involves curating and annotating audio data to enhance AI interactions globally. Candidates must have native Norwegian proficiency, a strong command of English...Hourly payRemote workFlexible hoursPantera Capital
United States4 days ago- Dorado is seeking expert Evaluators in program management / implementation planning to review AI-generated documents, spreadsheets, and slide decks for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise to grade outputs. This is a remote,...Remote jobHourly payWork at office
- A talent marketplace is seeking soccer experts to evaluate live soccer games. The role involves assessing AI-generated commentary, scoring performance, and providing feedback. Qualified candidates will have deep expertise in soccer, strong analytical and communication...Contract work
$50 per hour
Prolific seeks Product Designers and UX Specialists to help train AI models using your expertise. You'll evaluate AI-generated designs and ensure usability and accessibility while working from home. Ideal candidates hold a BS, MS, or PhD in a relevant field and have at...Remote jobWork from homeFlexible hours$24 per hour
Prolific is seeking an AI Trainer with advanced Tamil fluency to evaluate AI models' understanding of the Tamil language's emotional and cultural nuances. Responsibilities include assessing audio clips, reviewing tones for cultural relevancy, and ensuring quality control...Remote jobFlexible hours- Alignerr is seeking a Population Health Informaticist for AI training. This remote, hourly contract role requires 10-40 hours per week... ...to improve AI understanding of population health data. You will evaluate AI-generated analyses, identify errors, review data pipelines,...Remote jobHourly payContract work10 hours per weekFlexible hours
$43 - $47 per hour
...Role Overview Help improve how advanced AI models respond to sensitive topics in... ...language fluency and cultural judgment to evaluate model behavior and support safer, more reliable... ...guidelines to classify prompts and conversations. Identify adversarial wording and...Remote jobHourly payImmediate startSaidGig
Remote1 day ago$18 - $22 per hour
...Overview Help improve the safety of advanced AI models by applying Thai language fluency and cultural judgment to evaluate how models respond to sensitive topics in... ...structured guidelines to classify prompts and conversations. Identify adversarial wording and...Remote jobHourly payImmediate startSaidGig
Remote1 day ago- ...Audio Event Conversation Evaluator is a remote evaluation track for reviewing audio annotation prompts and responses against AuraOne's quality rubric... ...modeling team can use to retrain. Why this role matters AI data reviewers help turn audio annotation outputs into...Remote jobHourly payFor contractors10 hours per week
- ...AI Safety and Red-Team Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack... ...Score model defenses across single-turn and multi-turn conversations. Triage emerging attack vectors and route them to the...Remote jobHourly payFor contractors10 hours per week
$20 per hour
...detail-oriented professionals based in the United States to support AI model improvement through high-quality data review, annotation,... ..., you will work on structured AI data tasks used to train and evaluate the Gemini model. CONTRACT: Short-term freelance/contractor...Hourly payContract workTemporary workFor contractorsFreelanceRemote work$30 per hour
...Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous agent systems... ...stakeholders. Requirements Strong experience in LLM evaluation, AI output analysis, QA/testing, UX research, or similar analytical...Remote jobHourly payContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Tutor Conversation AI Evaluator [Remote]. Be the first to apply!
