AI Rubric Reviewer - Fully Remote | Up to $75/hr
$75 per hourMercor
- Remote job
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: AI Evaluation Rubric Reviewer
Type: Contract
Compensation: $40–$75/hour
Location: Remote
Role Responsibilities
- Review AI-drafted tasks and evaluate model attempts for fairness and accuracy.
- Assess if the grading rubric aligns with the prompt and correct any unfair criteria.
- Flag tasks that cannot be saved and provide concise feedback on each evaluation.
- Analyze long agent transcripts to identify errors and improve AI model performance .
- Ensure rubric structures are correct per guidelines and enhance training data quality .
Qualifications
Must-Have
- Substantial experience writing or reviewing rubrics for LLM evaluations .
- Ability to read long agent transcripts and identify errors.
- Precise writing skills and a commitment to data verification.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
$40 - $75 per hour
...AI Evaluation Rubric Reviewer is a remote evaluation track for reviewing ai evaluation rubric evaluation prompts and responses against AuraOne's quality... ...be authorized to work from US. Compensation $40–$75 / hr Application process Apply through AuraOne's specialist...Remote jobFor contractors10 hours per week- AuraOne, Inc. is seeking an AI Agent Interaction Specialist to remotely review ai agent interaction evaluation prompts and responses against AuraOne's quality rubric. You will compare paired outputs, label edge cases, and write structured feedback for the modeling team...Remote work10 hours per week
$50 per hour
prolificacademicltd is seeking a UX design reviewer to contribute professional product design and UX expertise to paid tasks that train and evaluate AI models. After a short skills assessment, successful applicants may join a network of independent participants and review...Remote jobFlexible hours$18 per hour
Classifieds24.info is seeking remote workers to help review AI-generated content, including voice models, images and videos. You will provide feedback to improve AI performance and ensure content meets project requirements. Strong written and verbal communication, attention...Remote jobWeekly payFlexible hours$18 per hour
Adver.info is seeking remote assistants to review and help improve AI systems. In this role you will interact with AI voice models and review AI-generated images and videos, providing feedback to enhance accuracy and usefulness. This is a flexible, remote, gig-style opportunity...Remote jobHourly payWeekly payFlexible hours$55 per hour
...technical talent with leading AI research labs. Headquartered in... ...Wholesale and Technical Sales Reviewer Type: Contract... ...$50–$55/hour Location: Remote Role Responsibilities Evaluate... ...performance. Develop evaluation rubrics and benchmarks for assessing...Remote workContract workSummer work$18 per hour
Globalclassifieds.info is seeking individuals to review AI content remotely. You will interact with AI voice models, review images and videos, and provide feedback to improve AI performance. The role offers remote work with flexible hours and weekly payments. Compensation...Remote jobHourly payWeekly payFlexible hours$18 per hour
Globalclassifieds is offering remote gigs to review AI content, including voice models and generated images/videos. Earn up to $18 per hour, with flexible scheduling and weekly payments. Candidates should demonstrate strong communication and attention to detail, and adapt...Remote jobHourly payWeekly payFlexible hours$20 - $45 per hour
Quant STEM Bachelor AI Training Reviewer (English) Review homework... ...accuracy and reasoning. This remote, part-time contractor... ...Hourly · $20-$45/hr $20-$45/hr Contract,... ...-user perspective in a fully remote, 16-week US project... ...remote, flexible role pays $75 per hour and suits...Remote workHourly payContract workPart timeFor contractors10 hours per weekFlexible hours- ...HR Policy Workflow AI Reviewer is a remote review track for evaluating AI outputs in policy review workflows. Reviewers grade citation accuracy, statutory... ...on real matters. Comfort applying multi-page rubrics consistently across long batches. Clear written reasoning...Remote jobHourly payFor contractors10 hours per week
$90 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco... .... Position: Finance & Insurance Practice Reviewer Type: Contract Compensation: $75–$90/hour Location: Remote Role Responsibilities Evaluate AI-...Remote jobContract workSummer workFlexible hours$80 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Vision Verticals Reviewer Type: Contract Compensation... ...$60–$80/hour Location: Remote Role Responsibilities... ...prompts, golden outputs, and rubrics. Evaluate the accuracy and...Remote jobContract workSummer work$120 per hour
...technical talent with leading AI research labs. Headquartered in... ...$80–$120/hour Location: Remote Role Responsibilities... ...against domain-specific quality rubrics. Identify factual, aesthetic... ...improve AI-generated content. Review documents, spreadsheets, and...Remote jobContract workSummer workWork at office$120 per hour
...technical talent with leading AI research labs. Headquartered in... ...$80–$120/hour Location: Remote Role Responsibilities... ...against domain-specific quality rubrics. Identify factual, aesthetic... ...****@*****.*** PS: Our team reviews applications daily. Please...Remote jobContract workSummer workWork at office$18 per hour
Globalclassifieds.info is hiring to help improve AI by reviewing voice models and AI-generated images and videos from a remote location. You’ll provide feedback to improve accuracy and quality. This flexible, part-time role pays up to $18/hour with weekly payments and requires...Remote jobWeekly payPart timeFlexible hours$18 per hour
Realtor.pm is seeking individuals to test and review AI content remotely. Earn up to $18 per hour by talking to AI, evaluating AI-generated visuals, and providing feedback to improve AI performance. This talent gig offers flexible hours, remote work, and weekly payments...Remote jobHourly payWeekly payFlexible hours- ...engagements, and advisory opportunities. We are seeking experienced HR professionals to help train a language model on professional... ...and slide decks, while advancing core HR workflows. This flexible, remote role offers 5-20 hours per week with ASAP start and a competitive...Remote jobHourly payImmediate startFlexible hours
$25 per hour
...the gig Join our distributed team as a Remote AI Task Reviewer and help improve AI systems by... ...when needed Perks & Benefits ~$25/hr + bonuses ~ Work from home on your... ...If you’re looking for a well-paid, fully remote side gig you can do from your own...Remote jobSecond jobImmediate startWork from homeFlexible hours- ...Assessment Rubric Learning Content Reviewer is a remote review track for evaluating AI outputs across assessment rubric learning content specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk;...Remote jobHourly payFor contractorsWork experience placement10 hours per week
$90 per hour
...technical talent with leading AI research labs. Headquartered in... ...Position: Civil Engineer Reviewer Type: Contract Compensation... ...: $90/hour Location: Remote Role Responsibilities... ...omissions. Evaluate grading rubrics for clarity and specificity, ensuring...Remote workContract workSummer work- Uplinq is seeking a Tax Reviewer to serve as the final technical check before returns go... ...review federal and state returns, including AI-prepared work, and use our Claude-based... ..., and proactive problem solving. This fully remote US role offers growth as the company scales...Remote job
- AuraOne is seeking reviewers for its remote Logistics & Planning Professionals track. You will assess AI outputs against supply chain workflows and firm policies, grading tone... ..., and stakeholder fit using a structured rubric. The role emphasizes clear written reasoning...Remote jobFor contractors
- ...seeking computer science specialists to evaluate AI-generated graphical abstracts derived from academic papers. This fully remote engagement focuses on expert judgement to... ...faithfully reflect the underlying research. You will review CS papers alongside AI-generated graphics,...Remote job
- AuraOne, Inc. is seeking a remote Differential Diagnosis AI Reviewer to evaluate evaluation prompts and responses against the company's quality rubric. You will compare paired outputs, label edge cases, and write structured feedback for model retraining. The role is an...Remote jobFor contractors10 hours per week
- AI Trainer Jobs in the United States seeks a remote Business Document Expert (German Speaker) to evaluate German generalist prompts and responses against AuraOne's rubric. You will label edge cases, provide structured feedback for retraining, and compare outputs to select...Remote workHourly payFor contractors
- AuraOne seeks experienced reviewers to evaluate car mounted 360 video recorder outputs against a versioned rubric, tagging severity and highlighting edge cases. You will compare... ...failure modes for model improvement. This remote contractor role requires clear reasoning...Remote jobFor contractors
- Uplinq Financial Technologies is seeking a Tax Reviewer who will be the final technical check before a return goes... ...estates, and nonprofits, including those prepared with AI, using Claude and our platform. Fully remote from the US. Ideal candidates have tax review...Remote job
- ...seeking a Federal Performance and Financial Audit Expert to remotely review AI outputs across audit workflows. You will grade calculations,... ...credentials (CPA/CFA/ACA or equivalent), and the ability to apply rubrics consistently. 10+ hours weekly async availability is...Remote job
- ...seeking an experienced forestry and land management expert to review AI-generated scenarios for accuracy and practicality. You will... ...providing structured feedback to improve AI reasoning. This is a fully remote, task-based contract offering 10-40 hours per week and...Remote jobContract work10 hours per weekFlexible hours
- ...Ltd. is seeking a qualified neuroscience expert for a fully remote verification role. You will compare AI-generated graphical abstracts to source papers,... ...deep knowledge of current methods, and a track record of meticulous review. #J-18808-Ljbffr prolificacademicltdRemote jobFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Rubric Reviewer - Fully Remote | Up to $75/hr. Be the first to apply!
- remote data architect New York, NY
- remote data manager New York, NY
- remote property manager New York, NY
- office manager remote New York, NY
- it manager remote New York, NY
- claims specialist remote New York, NY
- project manager remote europe New York, NY
- remote video game New York, NY
- remote office assistant New York, NY
- node js remote New York, NY


