Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Voice Evaluation Specialist [Remote]

$20 - $25 per hour

Innodata Inc.

Remote
  • Remote job

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

Scope of the Role: 

 

We are looking for detail-oriented Voice Specialists to evaluate the performance of AI models through real-time, voice-based conversations. In this role, you will interact with two different AI models using the same assigned scenario, carefully compare their responses, and provide structured evaluations based on defined quality criteria.

The goal of the evaluation is to create a fair, consistent comparison between models and identify which model delivers the stronger overall conversational experience.

What You’ll Own:

  • Review an assigned scenario and roleplay a natural conversation with two different AI models .
  • Conduct comparable conversations with Model A and Model B, keeping the scenario, conversational approach, and overall interaction as consistent as possible.
  • Maintain approximately the same number of conversational turns with each model to support a fair comparison.
  • Record your voice during each interaction and pay close attention to the quality and behavior of the model's responses.
  • Evaluate each model across five defined evaluation dimensions .
  • Identify and categorize relevant error clusters that may occur in the audio or conversation.
  • Compare the performance of Model A and Model B based on the evaluation criteria and your observations.
  • Select the model that demonstrated the stronger overall performance according to the defined evaluation framework.
  • Write a detailed rationale explaining your final preference, referencing specific examples and observations from both conversations.
  • Apply evaluation guidelines consistently across different scenarios and model interactions

You’ll Thrive in This Role If You Have

  • Bachelors Degree 
  • Strong attention to detail and ability to notice subtle differences in conversational quality.
  • Excellent listening and comprehension skills.
  • Strong written communication skills, particularly the ability to explain observations clearly and objectively.
  • Ability to follow detailed evaluation guidelines consistently.
  • Comfort speaking naturally and roleplaying different conversational scenarios.
  • Ability to compare two interactions fairly without allowing personal preferences to influence the evaluation.
  • Strong critical-thinking and analytical skills.
  • Reliability and consistency when completing structured evaluation tasks.
  • Familiarity with AI assistants, voice-based AI, or conversational systems

The expected hourly salary range for this position is $20-$25 p/hour, based on experience, skills, and qualifications.

 

Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at  

If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at  View email address on job-boards.greenhouse.io and consider reporting it to the FTC at  ReportFraud.ftc.gov .

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Voice Evaluation Specialist [Remote] in Remote vacancy
  •  ...Vietnamese Voice Actor - Freelance AI Trainer Project World Wide - Remote Are you an experienced Vietnamese...  ...cutting-edge AI tools, record and evaluate Vietnamese speech samples, and...  .... Job title: Vietnamese Voice Acting Specialist – AI Trainer Employment type: Contract... 
    Suggested
    Hourly pay
    Contract work
    For contractors
    Freelance
    Remote work

    Meridial

    New York, NY
    2 days ago
  • $30 - $70 per hour

    Crossing Hurdles is seeking a remote Voice Actor/Coach to analyze short speech recordings...  ...intonation, breath support, rhythm and tempo, and evaluate emotional tone and delivery. You will...  ...sound and recording quality to support AI training. #J-18808-Ljbffr Crossing Hurdles
    Suggested
    Remote job
    Hourly pay
    10 hours per week

    Crossing Hurdles

    New York, NY
    2 days ago
  • AuraOne, Inc. is hiring an English Voice Expert (US) to work remotely as an independent contractor. You will review English voice evaluation prompts and responses, applyingAuraOne's quality rubric and providing structured feedback to retrain the model. Responsibilities... 
    Suggested
    Remote job
    For contractors

    AuraOne Human Data

    New York, NY
    3 days ago
  • Meridial is seeking independent contractors with native-level fluency in one or more Hindi language varieties for voice and audio evaluation work. Expected tasks include recording voice content, reviewing samples, and cross-comparing spoken recordings against written text... 
    Suggested
    For contractors
    Work from home

    Meridial

    New York, NY
    4 days ago
  •  ...Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio...  ...microphone to ensure clean, reliable audio input during voice evaluations. We offer a pay range of $6-to-$65 per hour,... 
    Suggested
    Hourly pay
    For contractors
    Freelance
    Remote work

    Agency.com

    New York, NY
    13 hours ago
  • $8 - $65 per hour

     ...record short scripted reads. Think light voice acting/voiceover: clean diction,...  ..., with the exact rate determined after evaluating your experience, expertise, and geographic...  ...not apply. Job title: Thai Voice Acting Specialist - AI Trainer Employment type: Contract Workplace... 
    Hourly pay
    Contract work
    For contractors
    Remote work

    Remote

    San Francisco, CA
    1 day ago
  • Crossing Hurdles is seeking an Image Evaluation Expert for a remote contract role in the United States. You will upload photographs, compare real images with AI outputs, and assess lighting, perspective, and textures to ensure accuracy. You will provide clear, evidence-... 
    Remote job
    Contract work
    10 hours per week

    Crossing Hurdles

    New York, NY
    2 days ago
  • Logoslabs is seeking a Thai Voice Acting Specialist to support AI training projects. This contract, remote from the US, requires native fluency, clear pronunciation, and expressive delivery across multiple tones. You will record short scripted reads, participate in group... 
    Remote job
    Contract work

    Remote

    San Francisco, CA
    1 day ago
  • $8 - $65 per hour

    Agency seeks a Thai voice acting specialist for AI training. You will record short scripted reads with expressive, conversational delivery in a quiet space using a good mic. Responsibilities include recording in your dominant language, following directions, and delivering... 
    Remote job
    Hourly pay
    For contractors

    Agency

    New York, NY
    5 days ago
  • $60 - $70 per hour

     ...’re looking for experienced FP&A professionals to join a fully remote, project-based contract focused on financial analysis and AI evaluation. In this role, you’ll apply your expertise in financial modeling, forecasting, variance analysis, and financial reporting to evaluate... 
    Hourly pay
    Contract work
    For contractors
    Remote work
    Monday to Friday

    Truelancer

    New York, NY
    2 days ago
  • $60 - $70 per hour

     ...States is seeking an experienced FP&A professional for a fully remote, project-based contract focused on financial analysis and AI evaluation. You will apply your expertise in financial modeling, forecasting, variance analysis, and financial reporting to evaluate AI-generated... 
    Remote job
    Hourly pay
    Contract work
    For contractors

    Truelancer

    New York, NY
    2 days ago
  • $21 - $35 per hour

    Crossing Hurdles is seeking an Image Evaluation Expert (Visual Arts / Photography) on a remote contract basis. You will upload photos, compare AI-generated images with real photographs, and assess lighting, perspective, and textures to ensure visual accuracy. You will... 
    Remote job
    Hourly pay
    Contract work

    Crossing Hurdles

    New York, NY
    4 days ago
  • Softechub is seeking an Image Evaluation Expert to remotely support our AI-driven image assessment projects in the United States. You will rate photos, provide detailed feedback, and help train next-generation AI systems with high-quality real-world input. The role emphasizes... 
    Remote job

    Softechub

    New York, NY
    4 days ago
  • Cardiac Imaging Specialist — Remote AI Report Evaluation is a remote review track for evaluating AI outputs across cardiac imaging specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document... 
    Remote job
    Hourly pay
    For contractors
    Work experience placement
    10 hours per week

    AuraOne Human Data

    New York, NY
    3 days ago
  • agency seeks a practitioner to evaluate and improve language models' handling of practical video production questions. You will draw on cinematography, directing, lighting, sound, editing, and post-production to test realistic workflows and document where model responses... 
    Remote job
    For contractors

    agency

    New York, NY
    3 days ago
  • AuraOne, Inc. is seeking a Domain Contributor Evaluation Specialist for a remote contractor role in the United States. You will review AI outputs against workflows, policy, and stakeholder-fit using a structured rubric, and document the next steps for model training. Responsibilities... 
    Remote job
    For contractors
    10 hours per week

    AuraOne, Inc.

    New York, NY
    2 days ago
  • $21 - $35 per hour

     ...project's quality standards Compare your original images with the AI-generated versions Spot imperfections, mismatches and visual...  ...English, written and spoken No AI experience needed. Experience evaluating AI-generated images, annotating visual data, labeling data,... 
    Remote job
    Hourly pay
    Part time
    For contractors
    Flexible hours

    OpenTrain AI

    New York, NY
    1 day ago
  • $60 per hour

    Prolific is seeking Computer Science Specialists to join our Expert Network to train and evaluate AI models. You’ll complete a quick 10- to 15-minute test and, if successful, join Prolific as a paid participant working from home with flexible hours. Researchers pay for... 
    Remote job
    Work from home
    Flexible hours

    Prolific

    New York, NY
    4 days ago
  • $30 per hour

    Zekai is seeking a curious, clear writer to help evaluate AI assistants on realistic everyday tasks. You’ll craft short requests, interact with the AI over several turns, and rate the results against a professional standard. This is a 100% remote contract in the United... 
    Remote job
    Hourly pay
    Contract work
    10 hours per week
    Flexible hours

    Confidential

    New York, NY
    1 day ago
  • $45 - $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  .... Position: Behavioral Health Expert, AI Safety and Model Evaluation Type: Contract Compensation: $45–$70/hour Location... 
    Contract work
    Summer work
    Remote work
    Weekday work

    Mercor

    New York, NY
    14 days ago
  •  ...English Voice Coaching Evaluation Specialist is a remote evaluation track for reviewing english voice coaching evaluation evaluation prompts and responses...  ...team can use to retrain. Why this role matters AI data reviewers help turn english voice coaching evaluation... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    a month ago
  • agency is seeking a contractor to evaluate AI-generated Rust solutions, probing model reasoning across systems topics and documenting failures for improvement. You will assess asynchronous code, memory management, API integration, and secure coding practices while providing... 
    Remote job
    Hourly pay
    For contractors
    Freelance

    agency

    New York, NY
    3 days ago
  • OpenTrain AI, Inc. is seeking an Employment and Labor Law AI Evaluation Expert to review AI-generated answers to U.S. employment law scenarios, assess legal accuracy and reasoning, and write feedback to improve models. This 100% remote, part-time freelance role requires... 
    Remote job
    Part time
    Freelance

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • $30 - $40 per hour

    211 Specialist, Government Services AI Evaluator Use your experience helping people find public benefits and community resources to evaluate AI answers, write scoring guides, and give precise feedback. This U.S.-based contract role pays $30 to $40 per hour. Remote Hourly... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Flexible hours

    OpenTrain AI, Inc.

    New York, NY
    2 days ago
  •  ...About the role Meridial seeks independent Audio Evaluation Specialists for an AI benchmark evaluation project. You will design evaluation frameworks, create training data, and audit AI outputs for realistic customer support scenarios across travel, finance, and technical... 
    Hourly pay
    For contractors
    Freelance
    Work at office
    Remote work
    Worldwide

    remote work

    New York, NY
    13 hours ago
  • $187.2k

     ..., who manages all applications and next steps. Our partner is looking for an Industrial Plant Reliability & Maintenance Expert, AI Evaluation based in Switzerland. This role offers an opportunity to apply extensive industrial plant and maintenance expertise to an innovative... 
    Hourly pay
    Contract work
    Part time
    Remote work
    Flexible hours

    Jobgether

    Remote
    2 days ago
  • $187.2k

     ...About the Role Contribute your process-plant expertise to an AI evaluation project by reviewing a simulated industrial plant benchmark. Your feedback will help assess whether the simulated environment, sensor data, and maintenance scenarios reflect real-world operating... 
    Hourly pay
    Full time
    Contract work
    Remote work
    Flexible hours

    Clera

    Remote
    5 days ago
  • AuraOne is seeking an AI Agent Evaluator to remotely review and label ai agent evaluation prompts and responses against AuraOne's quality rubric. You will compare outputs, identify edge cases, and provide structured feedback to support retraining. The role focuses on auditing... 
    Remote job

    Auraone

    New York, NY
    3 days ago
  • Evaluate audio-based conversational AI in realistic customer support scenarios. Contribute to high-quality datasets for advanced AI models. Work remotely as a freelance independent contractor. Responsibilities: Design and conduct role-play scenarios across travel, finance... 
    Remote job
    Hourly pay
    For contractors
    Freelance
    Flexible hours

    Jobgether

    New York, NY
    1 day ago
  • Design and run realistic German customer service role-plays in travel, finance, and tech support. Evaluate AI audio responses using structured criteria for accuracy, naturalness, and task completion. Build high-quality audio datasets that reflect real-world customer expectations... 
    Remote job
    Hourly pay
    Freelance
    Flexible hours

    Jobgether

    New York, NY
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Voice Evaluation Specialist [Remote]. Be the first to apply!