AI Voice Evaluation Specialist [Remote]
$20 - $25 per hourInnodata Inc.
- Remote job
Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.
Scope of the Role:
We are looking for detail-oriented Voice Specialists to evaluate the performance of AI models through real-time, voice-based conversations. In this role, you will interact with two different AI models using the same assigned scenario, carefully compare their responses, and provide structured evaluations based on defined quality criteria.
The goal of the evaluation is to create a fair, consistent comparison between models and identify which model delivers the stronger overall conversational experience.
What You’ll Own:
- Review an assigned scenario and roleplay a natural conversation with two different AI models .
- Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible.
- Maintain approximately the same number of conversational turns with each model to support a fair comparison.
- Record your voice during each interaction and pay close attention to the quality and behavior of the model's responses.
- Evaluate each model across five defined evaluation dimensions .
- Identify and categorize relevant error clusters that may occur in the audio or conversation.
- Compare the performance of Model A and Model B based on the evaluation criteria and your observations.
- Select the model that demonstrated the stronger overall performance according to the defined evaluation framework.
- Write a detailed rationale explaining your final preference, referencing specific examples and observations from both conversations.
- Apply evaluation guidelines consistently across different scenarios and model interactions
You’ll Thrive in This Role If You Have
- Bachelors Degree
- Strong attention to detail and ability to notice subtle differences in conversational quality.
- Excellent listening and comprehension skills.
- Strong written communication skills, particularly the ability to explain observations clearly and objectively.
- Ability to follow detailed evaluation guidelines consistently.
- Comfort speaking naturally and roleplaying different conversational scenarios.
- Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation.
- Strong critical-thinking and analytical skills.
- Reliability and consistency when completing structured evaluation tasks.
- Familiarity with AI assistants, voice-based AI, or conversational systems
The expected hourly salary range for this position is $20-$25 p/hour, based on experience, skills, and qualifications.
Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at
If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at View email address on job-boards.greenhouse.io and consider reporting it to the FTC at ReportFraud.ftc.gov .
- ...Vietnamese Voice Actor - Freelance AI Trainer Project World Wide - Remote Are you an experienced Vietnamese... ...cutting-edge AI tools, record and evaluate Vietnamese speech samples, and... .... Job title: Vietnamese Voice Acting Specialist – AI Trainer Employment type: Contract...SuggestedHourly payContract workFor contractorsFreelanceRemote work
$30 - $70 per hour
Crossing Hurdles is seeking a remote Voice Actor/Coach to analyze short speech recordings... ...intonation, breath support, rhythm and tempo, and evaluate emotional tone and delivery. You will... ...sound and recording quality to support AI training. #J-18808-Ljbffr Crossing HurdlesSuggestedRemote jobHourly pay10 hours per week- AuraOne, Inc. is hiring an English Voice Expert (US) to work remotely as an independent contractor. You will review English voice evaluation prompts and responses, applyingAuraOne's quality rubric and providing structured feedback to retrain the model. Responsibilities...SuggestedRemote jobFor contractors
- Meridial is seeking independent contractors with native-level fluency in one or more Hindi language varieties for voice and audio evaluation work. Expected tasks include recording voice content, reviewing samples, and cross-comparing spoken recordings against written text...SuggestedFor contractorsWork from home
- ...Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio... ...microphone to ensure clean, reliable audio input during voice evaluations. We offer a pay range of $6-to-$65 per hour,...SuggestedHourly payFor contractorsFreelanceRemote work
$8 - $65 per hour
...record short scripted reads. Think light voice acting/voiceover: clean diction,... ..., with the exact rate determined after evaluating your experience, expertise, and geographic... ...not apply. Job title: Thai Voice Acting Specialist - AI Trainer Employment type: Contract Workplace...Hourly payContract workFor contractorsRemote work- Crossing Hurdles is seeking an Image Evaluation Expert for a remote contract role in the United States. You will upload photographs, compare real images with AI outputs, and assess lighting, perspective, and textures to ensure accuracy. You will provide clear, evidence-...Remote jobContract work10 hours per week
- Logoslabs is seeking a Thai Voice Acting Specialist to support AI training projects. This contract, remote from the US, requires native fluency, clear pronunciation, and expressive delivery across multiple tones. You will record short scripted reads, participate in group...Remote jobContract work
$8 - $65 per hour
Agency seeks a Thai voice acting specialist for AI training. You will record short scripted reads with expressive, conversational delivery in a quiet space using a good mic. Responsibilities include recording in your dominant language, following directions, and delivering...Remote jobHourly payFor contractors$60 - $70 per hour
...’re looking for experienced FP&A professionals to join a fully remote, project-based contract focused on financial analysis and AI evaluation. In this role, you’ll apply your expertise in financial modeling, forecasting, variance analysis, and financial reporting to evaluate...Hourly payContract workFor contractorsRemote workMonday to Friday$60 - $70 per hour
...States is seeking an experienced FP&A professional for a fully remote, project-based contract focused on financial analysis and AI evaluation. You will apply your expertise in financial modeling, forecasting, variance analysis, and financial reporting to evaluate AI-generated...Remote jobHourly payContract workFor contractors$21 - $35 per hour
Crossing Hurdles is seeking an Image Evaluation Expert (Visual Arts / Photography) on a remote contract basis. You will upload photos, compare AI-generated images with real photographs, and assess lighting, perspective, and textures to ensure visual accuracy. You will...Remote jobHourly payContract work- Softechub is seeking an Image Evaluation Expert to remotely support our AI-driven image assessment projects in the United States. You will rate photos, provide detailed feedback, and help train next-generation AI systems with high-quality real-world input. The role emphasizes...Remote job
- Cardiac Imaging Specialist — Remote AI Report Evaluation is a remote review track for evaluating AI outputs across cardiac imaging specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document...Remote jobHourly payFor contractorsWork experience placement10 hours per week
- agency seeks a practitioner to evaluate and improve language models' handling of practical video production questions. You will draw on cinematography, directing, lighting, sound, editing, and post-production to test realistic workflows and document where model responses...Remote jobFor contractors
- AuraOne, Inc. is seeking a Domain Contributor Evaluation Specialist for a remote contractor role in the United States. You will review AI outputs against workflows, policy, and stakeholder-fit using a structured rubric, and document the next steps for model training. Responsibilities...Remote jobFor contractors10 hours per week
$21 - $35 per hour
...project's quality standards Compare your original images with the AI-generated versions Spot imperfections, mismatches and visual... ...English, written and spoken No AI experience needed. Experience evaluating AI-generated images, annotating visual data, labeling data,...Remote jobHourly payPart timeFor contractorsFlexible hours$60 per hour
Prolific is seeking Computer Science Specialists to join our Expert Network to train and evaluate AI models. You’ll complete a quick 10- to 15-minute test and, if successful, join Prolific as a paid participant working from home with flexible hours. Researchers pay for...Remote jobWork from homeFlexible hours$30 per hour
Zekai is seeking a curious, clear writer to help evaluate AI assistants on realistic everyday tasks. You’ll craft short requests, interact with the AI over several turns, and rate the results against a professional standard. This is a 100% remote contract in the United...Remote jobHourly payContract work10 hours per weekFlexible hours$45 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... .... Position: Behavioral Health Expert, AI Safety and Model Evaluation Type: Contract Compensation: $45–$70/hour Location...Contract workSummer workRemote workWeekday work- ...English Voice Coaching Evaluation Specialist is a remote evaluation track for reviewing english voice coaching evaluation evaluation prompts and responses... ...team can use to retrain. Why this role matters AI data reviewers help turn english voice coaching evaluation...Remote jobHourly payFor contractors10 hours per week
- agency is seeking a contractor to evaluate AI-generated Rust solutions, probing model reasoning across systems topics and documenting failures for improvement. You will assess asynchronous code, memory management, API integration, and secure coding practices while providing...Remote jobHourly payFor contractorsFreelance
- OpenTrain AI, Inc. is seeking an Employment and Labor Law AI Evaluation Expert to review AI-generated answers to U.S. employment law scenarios, assess legal accuracy and reasoning, and write feedback to improve models. This 100% remote, part-time freelance role requires...Remote jobPart timeFreelance
$30 - $40 per hour
211 Specialist, Government Services AI Evaluator Use your experience helping people find public benefits and community resources to evaluate AI answers, write scoring guides, and give precise feedback. This U.S.-based contract role pays $30 to $40 per hour. Remote Hourly...Hourly payContract workPart timeFor contractorsRemote workFlexible hours- ...About the role Meridial seeks independent Audio Evaluation Specialists for an AI benchmark evaluation project. You will design evaluation frameworks, create training data, and audit AI outputs for realistic customer support scenarios across travel, finance, and technical...Hourly payFor contractorsFreelanceWork at officeRemote workWorldwide
$187.2k
..., who manages all applications and next steps. Our partner is looking for an Industrial Plant Reliability & Maintenance Expert, AI Evaluation based in Switzerland. This role offers an opportunity to apply extensive industrial plant and maintenance expertise to an innovative...Hourly payContract workPart timeRemote workFlexible hours$187.2k
...About the Role Contribute your process-plant expertise to an AI evaluation project by reviewing a simulated industrial plant benchmark. Your feedback will help assess whether the simulated environment, sensor data, and maintenance scenarios reflect real-world operating...Hourly payFull timeContract workRemote workFlexible hours- AuraOne is seeking an AI Agent Evaluator to remotely review and label ai agent evaluation prompts and responses against AuraOne's quality rubric. You will compare outputs, identify edge cases, and provide structured feedback to support retraining. The role focuses on auditing...Remote job
- Evaluate audio-based conversational AI in realistic customer support scenarios. Contribute to high-quality datasets for advanced AI models. Work remotely as a freelance independent contractor. Responsibilities: Design and conduct role-play scenarios across travel, finance...Remote jobHourly payFor contractorsFreelanceFlexible hours
- Design and run realistic German customer service role-plays in travel, finance, and tech support. Evaluate AI audio responses using structured criteria for accuracy, naturalness, and task completion. Build high-quality audio datasets that reflect real-world customer expectations...Remote jobHourly payFreelanceFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Voice Evaluation Specialist [Remote]. Be the first to apply!


