AI Evaluation Specialist
Virtual Vocations Inc
To support AI development, the part-time AI Evaluation Specialist will review and assess AI-generated outputs for quality and usability while collaborating with teams to refine evaluation standards in a remote contract role. Key responsibilities Review and critically assess AI-generated outputs for quality, clarity, and user experience Identify inconsistencies and improvement opportunities across various content types Provide insightful feedback to inform model improvements and enhance evaluation workflows Required qualifications Native English proficiency at C1 or C2 level Exceptional reading comprehension and attention to detail Strong observational skills with excellent aesthetic or editorial judgment Ability to think critically and deliver high-quality evaluations in ambiguous scenarios Excellent written and verbal communication skills
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the AI Evaluation Specialist in United States vacancy
$70 per hour
...clients Compensation: $70 per hour Join a cutting-edge AI research initiative focused on improving the quality, accuracy,... ...with exceptional critical thinking and communication skills to evaluate AI-generated responses across a variety of topics. In this role...SuggestedHourly payWeekly payContract workFor contractorsRemote workFlexible hoursWeekday AI
United States4 days ago$70 per hour
...AI Research Initiative Role Compensation: $70 per hour Join a cutting-edge AI research initiative focused on improving the quality... ...with exceptional critical thinking and communication skills to evaluate AI-generated responses across a variety of topics. In this...SuggestedHourly payContract workRemote workFlexible hoursWeekday
United States3 days ago- ...What Is An Ai Evaluation Specialist? An AI evaluation specialist assesses and tests artificial intelligence systems to ensure they perform accurately, reliably, and safely. They measure how well AI models complete tasks such as answering questions, generating content...SuggestedWork at officeRemote work
Valence
United States5 days ago - A tech company specializing in AI projects is seeking skilled LibreSprite users to assist in evaluating AI-generated visual content. As an independent contractor, you can work flexibly from anywhere, contributing around 5-20 hours per week depending on project needs. Ideal...SuggestedFor contractorsRemote work
Handshake
United States2 days ago - ...Join a pioneering AI initiative focused on building next-generation evaluation benchmarks for frontier AI models. We are seeking analytical and technically skilled professionals to identify where advanced AI systems fail in subtle, real-world scenarios. Working in a red...SuggestedFull timeContract workFor contractorsRemote workFlexible hours
Weekday
Remote22 days ago $25 - $30 per hour
...Bilingual Traditional Chinese AI Evaluation Specialist is a remote Chinese specialist track for evaluating chinese evaluation outputs against native-speaker standards. Reviewers spot fluency, register, and cultural-context errors that automated checks miss, and write...For contractorsRemote work10 hours per weekAuraOne Human Data
Remotea month ago- ...Join a fast-paced AI evaluation initiative supporting one of the world's leading AI research organizations. We are seeking detail-oriented professionals to evaluate AI-generated outputs by applying structured grading rubrics with precision and consistency. This is...Temporary workImmediate start
Weekday
Remote22 days ago - Mercor is seeking experienced music producers and audio engineers to evaluate generative music AI models. You will assess AI-generated tracks, rate musicality, creativity, and production quality, and label songs by genre and instruments. The role requires native Norwegian...Immediate startFlexible hours
$80 per hour
...Science Expert with Python experience for part-time, remote projects. This role focuses on tasks related to AI systems, including designing problems, evaluating solutions, and validating calculations. Ideal candidates will have a degree in Computer Science, proficiency...Remote jobHourly payPart time- OpenTrain AI seeks a Video Game AI Evaluation Expert on a contractor, part-time basis. You will design prompts for game development, esports, platforms, and communities; assess AI responses for accuracy, completeness and nuance; and develop evaluation datasets with clear...Remote jobPart timeFor contractors
$17 - $18 per hour
...you curious, detail-oriented, and excited about shaping the future of artificial intelligence? We\'re looking for AI Evaluation & Annotation Specialists to help train and improve Large Language Models (LLMs). In this role, you\'ll review AI-generated responses, provide...Hourly payShift work- A leading research accelerator is seeking a Geospatial Expert to enhance AI systems through advanced geospatial analysis. This entry-level, remote role involves evaluating geospatial datasets and supporting tasks aligned with crisis management and agriculture. Candidates...Remote jobContract work
$20 - $26 per hour
Prolific is seeking fluent Kannada speakers to act as evaluators who compare text and voice samples to assess naturalness and authenticity. You will listen to audio clips, rate quality, and flag any mismatches in tone or pronunciation, with emphasis on cultural context...Remote jobFlexible hours- Dorado is seeking language professionals for a six‑month, independent contractor engagement to contribute to an AI evaluation project. Review AI‑generated content, annotate language data, and provide feedback to improve accuracy and reliability. The role requires reliability...For contractors
$1,750 - $2,150 per month
Obsidian is looking for experienced cybersecurity professionals to review AI systems' threat detection and vulnerability assessments. Responsibilities include evaluating AI outputs and creating realistic cybersecurity scenarios. Ideal candidates should have over 3 years...- AuraOne is seeking a Physical Sciences Research Assistant for a Remote AI Evaluation track. You will review AI outputs in physics, reproduce key derivations, and document correct methods to help train modeling systems. This contractor-style role emphasizes rigorous reasoning...Remote jobPart timeFor contractors
- ...create role-play scenarios across domains such as travel, financial services, telecoms and technical support, contributing to diverse evaluation datasets. Responsibilities include evaluating performance with metrics on task completion, naturalness, audio comprehension, and...Remote jobContract work
- A leading AI research accelerator is seeking Geospatial Experts to enhance AI systems through evaluations and real-world applications. This entry-level contractor position is fully remote with flexible hours, primarily focusing on geospatial reasoning tasks. Responsibilities...Remote jobFor contractorsFlexible hours
- A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities...Immediate start
- Dorado is seeking Speech AI Evaluation Specialists to support AI content improvement. This freelance, part-time role is based remotely from Malaysia, with 10+ hours per week and a starting date immediately. You will evaluate Vietnamese-language responses and provide structured...Remote jobPart timeFreelanceImmediate start10 hours per week
$80 per hour
...part-time opportunity focused on quality assurance for autonomous AI agents. You will analyze complex systems, review tasks for logic... ...and detail-oriented skills, with experience in policy evaluation or logic puzzles preferred. Compensation can reach up to $80/hour...Part timeRemote workFlexible hoursMind Rift
United States1 day ago$150k - $250k
...About Distyl AI Distyl is an applied AI technology company partnering with the world’s most ambitious institutions to rearchitect... ...What We Are Looking For At Distyl, we build AI systems using Evaluation-Driven Development —an approach where evaluation is not an afterthought...Full timeWork at officeFlexible hours3 days per weekDistyl Ai
Remote1 day ago- ...Prolific is seeking Product Designers and UX Specialists to join our Expert Network, contributing to the training and evaluation of cutting-edge AI models. This role requires expertise in usability, design systems, and user research, providing essential feedback on design...Remote workWork from homeFlexible hours
Prolific
Tucson, AZ4 days ago - ...French Audio Evaluations Specialist - Freelance AI Trainer Project World Wide - Remote Project Overview We are sourcing independent Audio Evaluation Specialists for an AI benchmark evaluation project assessing advanced agentic audio models. As AI models increasingly...Hourly payFor contractorsFreelanceRemote work
Meridial
United States3 days ago - ...Prolific is seeking talented Product Designers and UX Specialists to join our Expert Network for training and evaluation of cutting-edge AI models. Candidates should have a relevant educational background and at least one year of experience in design-related fields. Responsibilities...Remote workWork from homeFlexible hours
Prolific
Arizona City, AZ5 days ago $20 - $60 per hour
...Help train next-generation AI systems by creating rigorous, real-world evaluations that test how well advanced models learn, reason, and perform. This remote contract opportunity is open to recent graduates, advanced-degree holders, and professionals from any background...Hourly payContract workFor contractorsRemote workSaidGig
Indiana3 days ago$60 - $85 per hour
...About the job Remote | Licensed Chemical Engineer & AI Evaluation Specialist - $60-$85/hour We are sharing a specialised part-time consulting opportunity for licensed US chemical engineers with professional experience in process design, process safety, plant operations...Hourly payFull timeContract workPart timeRemote work10 hours per weekFlexible hours24-MAG LLC
New York, NYa month ago- Rex.zone is seeking a remote, full-time Senior AI Data Annotation role supporting training-data quality for modern AI/ML systems. You will deliver high-precision human feedback across RLHF, LLM evaluation, prompt evaluation, and QA evaluation to drive measurable model performance...Remote jobFull time
- Welo Data is looking for a Data Labeling Associate in San Francisco to evaluate AI systems' handling of Arabic nuances. The role requires professional-level proficiency in Arabic and 2 years of AI safety experience. Responsibilities include critiquing Arabic AI outputs...
- ...Human Resources Expert for a remote contract in the United States. You will design realistic HR scenarios, craft model prompts, and evaluate AI-generated HR outputs while ensuring alignment with employment laws and best practices. You will collaborate with the team to...Remote jobContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Specialist. Be the first to apply!
