AI Agent Evaluator Remote Rubric & Feedback Specialist
AuraOne
AuraOne is seeking an AI Agent Evaluator to remotely review and label ai agent evaluation prompts and responses against AuraOne's quality rubric. You will compare outputs, identify edge cases, and provide structured feedback to support retraining.
The role focuses on auditing model outputs, labeling issues with rubric clauses, and calibrating scores weekly to maintain high reviewer quality. Applicants must be authorized to work remotely in the US.
#J-18808-LjbffrVacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the AI Agent Evaluator Remote Rubric & Feedback Specialist in United States vacancy
- AuraOne, Inc. is seeking a remote AI data reviewer to evaluate agent evaluation outputs against a versioned rubric. You will label edge cases, provide rationales, and help calibrate review quality across long batches. Ideal candidates have prior evaluation or annotation...Remote jobFor contractorsFlexible hours
- YO AI Labs seeks a former U.S. Consular Officer or Passport Specialist to evaluate AI-generated guidance on passport and citizenship... ...matters. This remote, contractor role asks... ..., developing rubrics for complex cases, and giving clear feedback to the team. #J-1880...Remote jobFor contractors
$80 - $120 per hour
...AuraOne is seeking a remote Pricing / ROI / revenue economics Evaluator to review prompts and model outputs against our quality rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model. As an independent contractor...Remote workHourly payFor contractors10 hours per weekAuraOne
United States4 days ago- Mercor seeks a contract evaluator for general business strategy to assess AI-generated artifacts against domain-specific rubrics. You will work remotely, evaluate quality, and provide structured written feedback to help improve AI model performance. You will collaborate...Remote jobContract work
- ...connects elite creative and technical talent with leading AI research labs. A remote contract role focuses on evaluating AI-generated artifacts against quality rubrics, identifying errors, and delivering structured feedback to support AI model improvements. Applicants should...Remote jobContract workWork at office
- ...Mercor is seeking a Software / AI / IT / data Evaluator for remote, contract work. You will assess AI-generated artifacts against quality rubrics, identify factual and presentation errors, and provide structured feedback to improve model outputs. Ideal candidates have...Remote workContract work
Mercor Inc
New York, NY13 hours ago - ...Role Title: AI Agent Evaluator Role Type: Contractor Location: U.S based Required Skills: ChatGPT Muse AI agents Attention... ...issues, satisfaction, and time Share clear written feedback on the agent experience Who we’re looking for: ~ US-based...Remote workContract workFor contractorsWork at officeWeekend work
Xperteez Technology Pvt Ltd
United States5 days ago $80 - $120 per hour
...Mercor is seeking a Software / AI / IT / data Evaluator for contract work. The role is remote and pays $80–$120 per hour. You will assess AI outputs against domain rubrics, identify errors, and provide structured feedback to improve quality. Requirements include 5+ years...Remote workHourly payContract workMercor Inc
San Francisco, CA4 days ago- ...Role Title: AI Agent EvaluatorRole Type: ContractorLocation: United StatesWe are seeking detail-oriented professionals to evaluate AI agents performing everyday tasks. You'll test two AI agents... ...their performance, and provide feedback to improve their reliability and user...Remote workWork at office
YO AI Labs
New York, NY4 days ago - ...HELP SHAPE THE FUTURE OF AI BY EVALUATING HOW WELL AI ASSISTANTS UNDERSTAND USERS, USE PERSONAL... ...DELIVER HELPFUL CONVERSATIONS. Location: Remote Type: Independent contractor / Part-... ...assistant and compare your experiences. Your feedback will help assess how effectively each...Remote workPart timeFor contractors
OpenKrill
New York, NY5 days ago - ...AuraOne seeks an experienced Senior Journalist / Writer for a remote evaluation track. You will review prompts and responses, compare paired outputs, label edge cases, and provide structured feedback for retraining. You’ll help turn outputs into auditable labels, rationales...Remote workHourly payFor contractors
AuraOne
New York, NY4 days ago - ...RWS Group is seeking a web-savvy evaluator to influence the future of AI search by testing client search results and providing feedback to refine data used across major search engines... ...platforms. The role supports a flexible, remote, part-time schedule and welcomes...Remote workPart timeFreelanceWork from homeFlexible hours
Dorado
United States2 days ago - AuraOne, Inc. is seeking an AI Product Evaluator for a remote contractor role. You will review prompts and model outputs, apply a versioned rubric, and provide structured feedback to retrain the model. The work involves tagging issues, labeling edge cases, and maintaining...Remote jobFor contractors10 hours per week
- Aitrainer is hiring an AI Agent Evaluator to test upcoming AI agents using your own phone from anywhere in the United States. You will compare... ...scoring, documenting steps, and providing actionable feedback to improve agent behavior. Applicants should have a Bachelor...
- ...IT Helpdesk Agent Evaluator is a remote evaluation track for reviewing... ...against AuraOne's quality rubric. Reviewers compare... ...kind of structured feedback the modeling team can... ...this role matters AI data reviewers help turn... ...Remote · Independent specialist contractor....Remote jobHourly payFor contractors10 hours per week
- AuraOne is seeking a Chinese Voice Coach for a remote evaluation track. You will review prompts and model outputs against our quality rubric, label edge cases, and provide structured feedback to retrain and improve the system. The role involves assessing model outputs,...Remote job
- RWS TrainAI is seeking a flexible part-time Search Quality Evaluator to join our remote team. You will assess how everyday searches are handled and provide feedback to improve AI-powered results for global users. Ideal for students or freelancers in Poland, with a strong...Remote jobHourly payPart timeFreelanceWork from homeFlexible hours
$70 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...Compensation: $70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses to... ...outputs . Provide structured written feedback to improve training data quality...Remote workContract workSummer workMercor
Seattle, WA5 days ago$80 - $120 per hour
...technical talent with leading AI research labs.... ...philanthropy / community programs Evaluator Type: Contract... ...0/hour Location: Remote Role... ...domain-specific quality rubrics. Identify factual, aesthetic... ...clear, structured written feedback to improve AI model...Remote workContract workSummer workWork at office- ...independent contractor to review Portuguese generalist evaluation outputs. You will apply a versioned rubric to each pair of model responses, assign severity... ...rubric application across multi-page batches, with remote US eligibility and flexible weekly hours. #J-18808-...Remote jobFor contractorsFlexible hours
- ...Student Feedback AI Evaluator is a remote review track for evaluating AI outputs across student feedback ai specialist operations workflows. Reviewers grade workflow correctness, policy... ..., and stakeholder fit on a structured rubric. Flag operational risk, missed escalations...Remote jobHourly payFor contractorsWork experience placement10 hours per week
- ...seeking freelance German-speaking annotators to evaluate AI-generated German content with a keen eye for details... ...real-world expectations, providing structured feedback to improve model performance. The role is remote, contract-based, and suitable for short-term engagements...Remote workContract workTemporary workFreelanceImmediate start
Risedatalabs
United States2 days ago - ...Editor for small businesses and creators. This remote contract role involves evaluating AI-generated video edits and reviewing briefs... .... Responsibilities include providing feedback to AI teams and developing evaluation rubrics. You will collaborate with clients to...Remote workContract work
Mercor Inc
United States3 days ago - ...Mercor is a remote contractor role focusing on evaluating AI-generated artifacts within real estate, hospitality, and... .... You will assess quality against rubrics, spot factual, presentation, and aesthetic... ...errors, and deliver structured feedback to help improve AI models. The...Remote workFor contractorsWork at office
Mercor Inc
United States3 days ago - ...Western Governors University (WGU) is seeking an Online English Teacher - Entry Level Evaluator to score student submissions using a rubric-based approach and provide detailed written feedback to support student progress. The role emphasizes accuracy, fairness, and timely...
Workoo Technologies
Cleveland, OH15 hours ago - ...Mercor is seeking an FP&A / corporate finance Evaluator to join our remote team. You will assess AI-generated artifacts against quality rubrics, identify errors, and provide structured feedback to drive accuracy and timeliness. The role requires 5+ years in FP&A or...Remote workContract workWork at officeFlexible hours
Mercor Inc
United States4 days ago - ...Obsidian is seeking expert Evaluators in Personal finance /... ...planning to review AI-generated documents, spreadsheets... ...and quality. This is a remote, hourly engagement... ...artifacts against rubrics, identify errors, and provide... ..., structured written feedback to improve future work....Remote workHourly pay
Obsidian
New York, NY5 days ago - ...Mercor is seeking an evaluator for nonprofit/philanthropy programs in a remote, contract role. You will assess AI-generated artifacts against quality rubrics, identify factual and presentation errors, and deliver concise feedback. Deep domain expertise will guide grading...Remote workContract workWork at office
Mercor Inc
San Francisco, CA4 days ago $50 per hour
...Spanish (Mexico) Audio Generalist Evaluator for a contract role. The position is remote and requires 10–20 hours per... ..., evaluate, and provide feedback to improve AI language tasks in Spanish and... ...The role focuses on developing rubrics, grading guidelines, and ensuring...Remote workContract work10 hours per weekMercor Inc
New York, NY3 days ago$25 - $45 per hour
...check your spam folder. This is a fully remote position open to candidates based in... ...States. Pay: $25$45/hr Role Title: AI Agent Evaluator Role Type: Contractor Location: U... ...and time # Share clear written feedback on the agent experience Who were looking...Remote workFull timeFor contractorsWork at officeWeekend work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Agent Evaluator Remote Rubric & Feedback Specialist. Be the first to apply!
Related searches
- bail bond agent United States
- work from home chat agent United States
- gate agent United States
- telemarketer - state farm agent team member United States
- executive protection agent United States
- agent assistant United States
- commissioning agent United States
- registered agent United States
- transfer agent United States
- money transfer agent United States