AI Prompt Quality Evaluator & Feedback Analyst
Claru
AI Trainer Jobs is seeking a detail-oriented evaluator to score prompt-response pairs from large language models. You will use structured rubrics to rate factual accuracy, instruction-following, and coherence across batches, typically 20–50 pairs per session. You will flag edge cases, document subtle failure modes, and contribute to fine-tuning data. Sessions are asynchronous with calibration meetings every two weeks, offering potential tracks in legal, medical, or coding domains. #J-18808-Ljbffr Claru
Vacancy posted 23 hours ago
Similar jobs that could be interesting for youBased on the AI Prompt Quality Evaluator & Feedback Analyst in New York, NY vacancy
- AuraOne, Inc. seeks a remote Historical Reasoning AI Evaluator to review prompts and responses, applying AuraOne's quality rubric. You will label edge cases, provide structured feedback, and inform retraining efforts through precise rationale. The role supports a contractor...QualityRemote jobFor contractors
- AuraOne is seeking an AI Agent Evaluator to remotely review and label ai agent evaluation prompts and responses against AuraOne's quality rubric. You will compare outputs, identify edge cases, and provide structured feedback to support retraining. The role focuses on auditing...QualityRemote job
- AI Trainer is a remote evaluation track for reviewing AI trainer evaluation prompts and responses against AuraOne’s quality rubric. Reviewers compare paired outputs, label edge cases, and write structured feedback for retraining the model. Responsibilities include evaluating...QualityRemote jobHourly payFor contractorsFlexible hours
$35 - $40 per hour
A leading evaluation organization seeks a Generalist Evaluator Expert for a remote, flexible... ...contract role. This position involves designing prompts for language model evaluations, defining... ...thinking skills, familiarity with AI tools like ChatGPT, and a BS or BA degree...QualityRemote jobContract workFlexible hours- A leading AI evaluation company is looking for candidates to assess AI-generated responses and provide structured feedback. This role involves evaluating reasoning quality, tone, and factual accuracy while supporting the enhancement of large language models. Candidates...QualityFull timeContract workPart timeRemote workFlexible hours
- AuraOne is seeking an AI Training Expert for a fully remote, US-eligible contractor role. You will review AI training evaluation prompts and outputs, label edge cases, and provide structured feedback to help retrain the model. Responsibilities include evaluating model outputs...Remote jobFor contractors
- ...Join a 16-week research study evaluating AI assistants from the perspective of everyday... ..., Instinct, or Muse and can assess prompts and responses in English. A part-time... ...everyday user. Assess response quality and provide feedback following the study’s guidelines....QualityFull timePart timeImmediate start
Braintrust
New York, NY3 days ago - ...Job-Senior Business Analyst & Prompt Engineer (AI Skills) Job Location - NYC, NY Job Type- Full Time... ...prompts, instructions and evaluation sets that let AI agents do work across... ...Skills with clear inputs, outputs, quality criteria and escalation points. Write...QualityFull time3 days per week
Maveric Systems Limited
New York, NY9 hours ago - AuraOne, Inc. is seeking a remote Browser Automation Task Evaluator to review prompts and responses against our quality rubric. This contractor role focuses on labeling edge cases, writing structured feedback, and helping retrain our models. You will compare paired outputs...QualityRemote jobFor contractors10 hours per week
- AuraOne, Inc. is seeking an AI Data Reviewer for the Misinformation Context AI Evaluator track. This remote contractor... ..., and providing structured feedback to retrain prompts. You will compare paired responses... ...to maintain high reviewer quality. Applicants must be able to...QualityRemote jobFor contractors
- AI Trainer Jobs seeks a Hindi Language Expert for a remote evaluation track. Review Hindi prompts and responses, compare paired outputs, label edge... ..., and provide structured feedback to retrain models. You will... ...to maintain reviewer quality. This contractor role offers...QualityRemote jobFor contractorsFlexible hours
- AuraOne, Inc. seeks a Revenue Recognition AI Reviewer to remotely assess evaluation prompts and responses against a published quality rubric. You will compare paired outputs, label edge cases, and produce structured feedback to retrain the model. As a contractor, you’ll...QualityRemote jobFor contractors10 hours per week
- Food Delivery AI Response Evaluator is a remote evaluation track for reviewing food... ...delivery ai response evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired... ...write the kind of structured feedback the modeling team can use to retrain...QualityHourly payFor contractorsRemote work10 hours per week
- Create challenging prompts and evaluate large language model responses for accuracy... ...where they fall short. Your feedback will focus on whether... ...execution to assess how well AI handles management-related topics... ..., relevance, and overall quality. Find factual errors,...QualityContract workPart timeFor contractorsRemote workWorldwide
- Misinformation Context AI Evaluator is a remote evaluation track for reviewing... ...misinformation context ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired... ...and write the kind of structured feedback the modeling team can use to...QualityHourly payFor contractorsRemote work10 hours per week
- AuraOne, Inc. is seeking a diligent remote contractor to evaluate at-work video recorder outputs against a versioned rubric. You will assign... ...tags, write rationales, and help identify edge cases for quality control. Responsibilities include comparing responses, labeling...QualityRemote jobFor contractors
$70 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...Remote Role Responsibilities Evaluate AI-generated responses to strengthen... .... Provide structured written feedback to improve training data quality and downstream performance. Develop...QualityContract workSummer workRemote work- ...is seeking a Luganda Language Expert for a remote evaluation track to review Luganda prompts and model outputs against a quality rubric. Reviewers compare paired outputs, label edge cases, and draft structured feedback for retraining. As a remote independent contractor...QualityRemote jobFor contractors
- AI Trainer Jobs is seeking remote contractors to review Application... ...Photoshop on Windows - Generalist prompts and responses, evaluating them against AuraOne's quality rubric. Reviewers compare paired... ...cases, and provide structured feedback to retrain the model. As a reviewer...QualityRemote jobFor contractors
- Management Consultant — AI Workflow Evaluator is a remote evaluation track for reviewing... ...ai workflow evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired... ...and write the kind of structured feedback the modeling team can use to retrain...QualityHourly payFor contractorsRemote work10 hours per week
- HELP SHAPE THE FUTURE OF AI BY EVALUATING HOW WELL AI ASSISTANTS UNDERSTAND... ...Anyone AI, we create high-quality training and evaluation data... ...compare your experiences. Your feedback will help assess how... ...needs. WHAT YOU'LL DO Write prompts about questions, interests,...QualityPart timeFor contractorsRemote work
- ...Title: Gmail & Google Calendar AI Assistant Evaluator Role Type: Contractor,... ...project enhancing AI assistant quality through real-world task... ...responses, providing structured feedback that improves model... ...projects. Familiarity with prompt writing and a practical understanding...QualityContract workFor contractorsRemote work
$137.9k - $256.1k
Sr. Legal AI Prompter / EvaluatorJob DescriptionSr... .... Legal AI Prompters / Evaluators are responsible for... ...by writing and refining prompts, skills, and agent configurations... ...into practical, high-quality AI outputs.Key... ...and provide strategic feedback to software engineers and...QualityFull timeWork at officeLocal areaFlexible hoursThomson Reuters
New York, NY2 days ago- Appsierra Group is seeking an AI Session Annotator for a fully remote, independent contractor role. You will review recorded AI sessions... ..., compare responses to visuals, and provide precise written feedback using a detailed rubric. Strong English, attention to detail,...Remote jobWeekly payFor contractorsFlexible hours
$14.5 per hour
...Join to apply for the AI Web Search Evaluator role at Welo Data Welo Data works with technology... ...to provide datasets that are high-quality, ethically sourced, relevant, diverse,... ...they meet user expectations. Offer feedback and recommendations to improve algorithm...QualityHourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hoursWelo Data
New York, NY4 days ago$35 per hour
AI Trainer Jobs in the United States is seeking detail oriented analysts to evaluate and rank AI generated responses to text prompts. You will judge answers on accuracy, relevance, clarity, conciseness, safety, localization, and how well they follow instructions, then...Remote jobHourly payFor contractorsFlexible hours- ...is seeking a Conlang Language Expert to remotely review language evaluation prompts and responses against our quality rubric. You will compare paired outputs, label edge cases, and write structured feedback used to retrain models. As a contractor, you will work...QualityRemote jobFor contractors10 hours per weekFlexible hours
$21 - $65 per hour
Photographer AI Image Analyst - $21-$65/hr Contract — Remote This... ...demand for skilled human evaluators who can assess visual quality, artifacts, and realism... ...moderation, creative QA, and prompt engineering.... ...provide detailed written feedback. No prior experience in...QualityHourly payExtra incomeContract workTemporary workFor contractorsFreelanceRemote workFlexible hours$65 - $70 per hour
...DESCRIPTION Business Analyst is responsible for... ...applications, AI-supported... ...owners, gather their feedback, and ensure business... ...and to develop and evaluate the technical requirements... ...implemented. - Quality assurance... ...user story issues promptly, and follow up on...QualityHourly payContract workFor contractors- ...thinking tech company in the United States is seeking an individual to evaluate AI model responses for personalization quality. Responsibilities include analyzing AI outputs, designing multi-turn prompts, and collaborating with teams to enhance model performance. Ideal...QualityRemote jobFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Prompt Quality Evaluator & Feedback Analyst. Be the first to apply!
Related searches
- evaluator New York, NY
- work from home web search evaluator New York, NY
- program evaluator New York, NY
- clinical evaluator New York, NY
- nurse evaluator New York, NY
- work from home social media evaluator New York, NY
- ai evaluator New York, NY
- education evaluator New York, NY
- quality evaluator New York, NY
- social media evaluator New York, NY