AI Response Evaluator & Quality Annotator
Blueprint
Blueprint is seeking an English-language AI Response Labeler / Annotator to evaluate the quality of AI-generated responses. The role involves comparing outputs from different AI models, judging based on accuracy, relevance, completeness, reasoning, and safety, and applying guidelines consistently across a high volume of tasks. Strong English proficiency and attention to detail are essential. You will participate in training, calibration, and ongoing quality reviews, with a focus on producing #J-18808-Ljbffr Blueprint
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Response Evaluator & Quality Annotator in New York, NY vacancy
- Anyone AI is seeking a QUALITY EVALUATOR - ENGLISH to help shape the future of AI by evaluating and comparing responses from advanced AI models. This remote, part-time contractor role offers up to 3 months of work and compensation in the $7-$12/hour range. You’ll interact...QualityRemote jobPart timeFor contractors
- Obsidian is seeking an individual to assess Telugu AI-generated responses, identifying strengths and areas for improvement. Applicants must... ...The role requires conducting fact-checks and generating quality evaluation data. Success comes from identifying inaccuracies and...QualityContract work
- Anyone AI is seeking a QUALITY EVALUATOR - ENGLISH to help shape the future of AI by evaluating and comparing responses from advanced AI models. This remote, part-time independent contractor role lasts up to 3 months and pays $7-$12 per hour. You’ll work with English-language...QualityRemote jobHourly payPart timeFor contractors
- Anyone AI is seeking a Quality Evaluator (English) to join a remote, independent contractor role. You will compare AI model responses, judge quality on clarity, accuracy, and usefulness, and provide well-reasoned explanations of your evaluations. The role requires excellent...QualityRemote jobPart timeFor contractorsFlexible hours
- Handshake is seeking Bangla speakers to help evaluate and rank AI responses, enabling future models to tailor answers for Bangla-speaking users. This is a remote, part-time, ongoing project you can take on alongside your current role. You will compare AI-generated responses...QualityRemote jobPart timeFlexible hours
- AuraOne, Inc. is seeking an AI Data Reviewer for the Misinformation Context AI Evaluator track. This remote contractor role... ...prompts. You will compare paired responses, tag misinfo and unsafe content... ...to maintain high reviewer quality. Applicants must be able to work...QualityRemote jobFor contractors
- Obsidian is looking for expert Evaluators to review AI-generated work products in Public-sector procurement and RFI response. In this remote hourly role, you will assess the accuracy and quality of documents, spreadsheets, and slide decks, applying your deep subject-matter...QualityRemote jobHourly payWork at office
- AuraOne, Inc. is seeking a Management Consultant — AI Workflow Evaluator as a remote contractor to assess AI workflow evaluation outputs against a quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain models. The...QualityRemote jobFor contractors
- RWS Group is seeking a Speech AI Evaluation Specialist to support the improvement of AI-generated content in Chinese... ...voice conversations, follow prompts, evaluate responses for relevance, accuracy, clarity, and overall quality, and provide objective ratings according to...QualityRemote jobPart timeFreelanceImmediate startWork from home10 hours per weekFlexible hours
- Anyone AI Inc. is seeking multilingual coding experts to evaluate AI-generated responses on a part-time remote contract lasting up to 3 months. This opportunity supports frontier... ...content for clarity, correctness, and quality, validating JSON structure and giving actionable...QualityRemote jobContract workPart time
- ...native Korean speaker to remotely evaluate healthcare front desk conversations with an AI, role-playing as a patient and... ...You will test AI understanding, responses, and action calls, then provide... ...and a quiet home workspace for quality contributions to AI research. #J...QualityRemote jobContract workFlexible hours
- ...review and label model outputs against a quality rubric, providing structured feedback to improve AI data quality. This is a remote evaluation track with US eligibility and an hourly... ...interview. You will compare paired responses, name issues with rubric references, tag...QualityRemote jobHourly payContract work
- Mercor is seeking experienced musicians to evaluate generative music AI models, working in Bengali and English. You will assess AI-... ...lyrics across genres and rate them against detailed quality standards. Responsibilities include comparing lyrics to published songs, rating...QualityPart timeImmediate startFlexible hours
- Mercor is seeking experienced musicians to evaluate generative music AI models, in partnership with a leading AI lab... ...of genres and rate it against detailed quality standards, working in Hindi and English. Responsibilities include comparing AI-generated lyrics to published...Quality
- Anyone AI is seeking multilingual evaluators to assess AI-generated responses for clarity, usefulness, and JSON accuracy. This independent contractor, part-time role is remote and can run up to 3 months, with flexible hours. You’ll review customer-service content, verify...QualityRemote jobPart timeFor contractorsFlexible hours
- Anyone AI is seeking multilingual coding experts to evaluate AI-generated responses. This part-time remote contract lasts up to 3 months and supports frontier AI model improvement... ...content for clarity, correctness, and quality. Validate JSON structure and provide clear...QualityRemote jobContract workPart timeFor contractors
$24 per hour
Prolific is seeking an AI Trainer with advanced Tamil fluency to evaluate AI models' understanding of the Tamil language... ...emotional and cultural nuances. Responsibilities include assessing audio clips,... ...relevancy, and ensuring quality control in audio outputs. The role...QualityRemote jobFlexible hours- Misinformation Context AI Evaluator is a remote evaluation track for reviewing misinformation context ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the...QualityHourly payFor contractorsRemote work10 hours per week
- Help shape the future of AI by evaluating the technical accuracy, language quality, and usefulness of AI-generated responses. - Location: Remote - Type: Independent contractor / Part-time - Project duration: up to 3 months ABOUT THE ROLE At Anyone AI, we create high...QualityPart timeFor contractorsRemote work
- Obsidian is hiring expert Evaluators in real estate, hospitality, and events to review AI-generated work products for accuracy and quality. This is a remote position requiring strong... ...Microsoft Office and Google Workspace. Responsibilities include evaluating documents and...QualityRemote jobWork at office
$30 per hour
...Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and... ...systems using defined rubrics and quality standards. Review multi-step agent workflows... ...Strong experience in LLM evaluation, AI output analysis, QA/testing, UX...QualityRemote jobHourly payContract work- Brazilian Portuguese AI Evaluator is a remote evaluation track for reviewing portuguese generalist evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling...QualityHourly payFor contractorsRemote work10 hours per week
- Medical Translation AI Evaluator is a remote clinical-review track for evaluating AI outputs... ...adherence in regulated workflows. Responsibilities Review AI outputs against current clinical... ...guidelines. Maintain reviewer-quality scores in weekly inter-rater calibration...QualityHourly payFor contractorsRemote work10 hours per week
- Learning Progression AI Evaluator is a remote review track for evaluating AI outputs across... ...whether a task actually gets done. Responsibilities Review AI outputs against current learning... ...or firm guidance. Maintain reviewer-quality scores in inter-rater calibration...QualityHourly payFor contractorsWork experience placementRemote work10 hours per week
- Materials Science AI Evaluator is a remote review track for evaluating AI outputs across... ...right method alongside the wrong one. Responsibilities Review AI outputs against current materials... ...standards. Maintain reviewer-quality scores in inter-rater calibration cycles...QualityHourly payFor contractorsRemote work10 hours per week
$13 per hour
...Argentina | Belize | Bolivia + 15 more Help evaluate how an AI assistant uses information from... ...Canada, outside Quebec, who can review AI responses carefully and explain what works and... ..., annotation, content review or quality assurance experience is helpful but not...QualityHourly payFor contractorsRemote work$18 per hour
Description: Interact with an AI assistant in Telugu. Translate English prompts... ...interact with the AI application. Evaluate AI-generated responses according to project guidelines.... ...accuracy, naturalness, localization quality, and cultural appropriateness. Provide...QualityHourly payTemporary workFreelanceImmediate startRemote workFlexible hours$18 per hour
AI Linguistic Evaluator - Telugu Dallas, Texas RWS - TrainAI / Temporary/Contract / Remote Freelance... ...skills and a keen eye for language quality? Join a remote AI project where you’ll... ...the quality of AI-generated responses according to project guidelines. Assessing...QualityRemote jobTemporary workFreelanceImmediate startFlexible hours$14.5 per hour
Join to apply for the AI Web Search Evaluator role at Welo Data Welo Data works with technology... ...companies to provide datasets that are high-quality, ethically sourced, relevant,... ..., and a passion for quality. Key Responsibilities Analyze search result performance and...QualityHourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hours- ...outcomes by connecting strategy and execution across AI, cloud, data, product development, and emerging technology... ...the Role We're looking for an English-language AI Response Labeler / Annotator to evaluate the quality of AI-generated responses. This role calls for strong...Quality
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Response Evaluator & Quality Annotator. Be the first to apply!
Related searches
- program evaluator New York, NY
- quality evaluator New York, NY
- evaluator New York, NY
- social media evaluator New York, NY
- education evaluator New York, NY
- ai evaluator New York, NY
- clinical evaluator New York, NY
- web search evaluator New York, NY
- ads evaluator New York, NY
- work from home web search evaluator New York, NY