Remote AI Product Evaluator — Rubric-Based Reviews
AuraOne, Inc.
New York, NY
- Remote job
AuraOne, Inc. is seeking an AI Product Evaluator for a remote contractor role. You will review prompts and model outputs, apply a versioned rubric, and provide structured feedback to retrain the model. The work involves tagging issues, labeling edge cases, and maintaining calibration with gold-standard examples each week. The role emphasizes clear written reasoning, attention to detail, and async availability of at least 10 hours per week. #J-18808-Ljbffr AuraOne, Inc.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Remote AI Product Evaluator — Rubric-Based Reviews in New York, NY vacancy
- ...AI Product Evaluator is a remote evaluation track for reviewing ai product evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write... ...output evaluation Rubric-based annotation Severity tagging...Remote jobHourly payFor contractors10 hours per week
$50 per hour
AuraOne, Inc. is seeking a Remote Evaluation Reviewer to assess Korean generalist outputs against AuraOne's quality rubric. You will review paired responses, apply severity tags, and provide structured feedback to help retrain the model. This contractor role emphasizes...Remote jobHourly payFor contractorsFlexible hours- Mercor is seeking a Product launch / experiment readiness Evaluator for a contract, remote role. You will evaluate AI-generated artifacts against rubrics, provide structured feedback, and apply domain expertise to grade outputs. Remote work with asynchronous deadlines....Remote jobContract workWork at office
- AuraOne, Inc. is seeking a remote contractor to review Australian voice actor evaluation outputs and apply a versioned rubric. You will label edge cases, provide structured feedback, and calibrate scores against gold-standard examples. The role requires strong attention...Remote jobFor contractors10 hours per week
- AuraOne, Inc. is seeking a remote AI data reviewer to evaluate agent evaluation outputs against a versioned rubric. You will label edge cases, provide rationales, and help calibrate review quality across long batches. Ideal candidates have prior evaluation or annotation...Remote jobFor contractorsFlexible hours
- ...elite creative and technical talent with leading AI research labs. We seek a contract Product Management / Roadmap / PRD Evaluator who can work remotely and autonomously to assess AI outputs against high-quality rubrics. Required are 5+ years in product management, native...Remote jobContract workWork at office
$50 - $70 per hour
Mercor is seeking a Generalist Expert to review and assess documents and AI-generated outputs. This contract-remote role requires a US/Canada base and strong written English, with flexible hours and independence across docs, slides, and sheets. Compensation ranges from...Remote jobHourly payContract workFlexible hours- AuraOne, Inc. is seeking a remote contractor to review AI outputs for Mexican ChatGPT User operations, ensuring policy adherence... ...train the modeling team with clear, policy-based feedback. Responsibilities include applying rubrics, calibrating scores, and maintaining inter-...Remote jobFor contractors
- ...Medical Expert, Spanish (Spain) for AI Clinical Review. This remote contractor role evaluates AI outputs, grades differential... ...guidelines, apply a structured rubric, and work asynchronously for at least 10 hours per week from a US-based remote setup. Licensing or board...Remote jobFor contractors10 hours per week
$54.6k
...quality reference evaluations are essential... ...Works You will review a series of de-... ...through a secure remote platform. For... ...structured scoring rubric. The workflow... ...scoring rubrics based on your findings... ...human experts for AI. Researchers, AI labs, and product teams use Terac...Remote workContract work- Mercor partners with a leading AI research organization to... ...criteria for real-world digital product design work. You will specify... ...will apply structured, evidence-based judgment to both AI-generated... ...iterating quickly on feedback from senior reviewers. #J-18808-Ljbffr Mercor
$32.76k
...specialists to evaluate clinical cases and... ...build scoring rubrics for diagnostic benchmarks... ...You will review a series of de-... ...through a secure remote platform. For... ...scoring rubrics based on the provided... ...human experts for AI. Researchers, AI labs, and product teams use Terac...Remote workContract work$70 - $85 per hour
...Remote - US-based / Flexible ET hours For a complete understanding of this... ...through execution, shaping products and journeys that remove friction... ...fast with discipline - Use AI as a force multiplier to... ...complex client work. We review portfolios first. Experience...Remote workLong term contractFor contractorsWork from homeFlexible hoursVariate
Baltimore, MD19 hours ago$120 per hour
...talent with leading AI research labs. Headquartered... ...Location: Remote Role Responsibilities Review published legal-... ...prompts and grading rubrics. Complete structural... ...source documents. Evaluate purchase and sale... ...resume AI interview based on your resume...Remote workContract workSummer workMercor
Chicago, IL25 days ago$60 - $100 per hour
...Finance Expert — AI Model Reviewer (Python & Financial Modeling) $60-100/hr Remote Freelance STEM About the... ...instructions and scoring rubrics, then roll up your... ...Candidates must have strong US-based financial knowledge... ..., or clarity gaps Evaluate whether AI outputs...Remote workHourly payOngoing contractContract workFreelanceFlexible hoursAlignerr
United States5 days ago$30 per hour
.../hour Location: Remote Commitment: 10-4... ...Responsibilities Evaluate outputs from... ...systems using defined rubrics and quality standards. Review multi-step agent... ...refinement and product improvements.... ...LLM evaluation, AI output analysis,... ...Proficiency in rubric-based scoring,...Remote jobHourly payContract work- OpenTrain AI, Inc. is seeking a remote contractor to evaluate language-model responses in social scenarios and provide... .... The work involves careful review, evidence-based decisions, and consistent use... ...model outputs, apply evaluation rubrics, document evidence, and explain...Remote jobFor contractors
$30 - $40 per hour
OpenTrain AI, Inc. is seeking a Passport Application AI Evaluation Expert for a remote, short-term contract. You will review passport questions for realism, create rubrics for accurate answers, and define evaluation... ...20-30 hours weekly, US-based availability, and...Remote jobHourly payTemporary work- ...zone is seeking a Senior AI data annotator to perform data labeling and evaluation for NLP tasks, RLHF... ...performance. This is a US-based remote, full-time role... ...You will apply strict rubrics, document edge cases,... ...collaborate asynchronously with reviewers and leads. A mid-...Remote jobFull time
- Obsidian is hiring expert Evaluators in Investment analysis / valuation / credit to review AI-generated work products for accuracy and quality. This remote, hourly position requires deep subject-matter expertise and professional fluency in English to provide structured...Remote workHourly payWork at office
$187.2k
...benchmark medical AI systems. This work... ...Works You will review a series of de-identified... ...and pharmaceutical product images through our remote platform. For each... ...develop scoring rubrics for AI evaluation. The workflow involves... ...scoring rubrics based on standard clinical...Remote workContract work- AuraOne is seeking a remote, contract-based bilingual Benefits Counselor (Spanish) to review and label model outputs against a quality rubric, providing structured feedback to improve AI data quality. This is a remote evaluation track with US eligibility and an hourly pay...Remote jobHourly payContract work
$150 per hour
...UK-Based UX/UI Product Designers is a remote review track for evaluating AI outputs across ui / ux design workflows. Reviewers grade craft, constraints, and tooling adherence... ..., and production-readiness on a structured rubric. Flag tooling-specific failures (file format...Remote workFor contractors10 hours per weekAuraOne Human Data
Remotemore than 2 months ago$120 per hour
...technical talent with leading AI research labs.... ...$120/hour Location: Remote Role Responsibilities Review published legal-benchmark... ...task prompts and grading rubrics. Complete structural QC... ...Upload resume AI interview based on your resume Submit form...Remote workContract workSummer workMercor
Chicago, IL25 days ago$125 per hour
The Work You will review medical materials and AI-generated outputs for... ...project guidelines. Evaluate AI outputs involving... ...writing. Apply project rubrics consistently across... .... Work arrangement: Remote contractor tasks.... ...project earnings: $15,000 based on the listed...Remote workHourly payPart timeFor contractors$60 per hour
...technical talent with leading AI research labs.... ...Dorsey . Position: Legal Reviewer — Document QC Type:... ...$60/hour Location: Remote Role Responsibilities Create rubrics to grade legal documents... ...Upload resume AI interview based on your resume Submit...Remote workContract workSummer workMercor
Chicago, ILa month ago- ...safeguarding and refining AI systems. This role centers on evaluating clinical... ...position is remote until further... ...Structured Grading Reviewing prompt-response... ...-dimensional rubrics (such as Likert... ...products; (v) charged, or... ...estimates the base compensation for...Remote workPart timeFor contractorsWork at officeLocal areaImmediate startMonday to FridayFlexible hours
$25 per hour
...Description About the gig \n Join our distributed team as a Remote AI Task Reviewer and help improve AI systems by completing short, style... ...cases and leave brief notes \n Work independently with chat-based support when needed \n Perks & Benefits \n \n ~$25/hr +...Remote jobSecond jobImmediate startWork from homeFlexible hoursRLDG
Lexington, MI16 days ago- ...Assessment Rubric AI Evaluator is a remote review track for evaluating AI outputs across assessment rubric ai specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next...Remote jobHourly payFor contractorsWork experience placement10 hours per week
- ...Secure Code Review AI Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios... ...mode, and pair each successful jailbreak with the rubric clause it violated so the safety team can patch the gap....Remote jobHourly payFor contractors10 hours per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote AI Product Evaluator — Rubric-Based Reviews. Be the first to apply!
Related searches
- software engineering manager remote New York, NY
- remote recruitment consultant New York, NY
- remote data manager New York, NY
- business intelligence analyst remote New York, NY
- benefits specialist remote New York, NY
- junior angular developer remote New York, NY
- remote video game New York, NY
- remote typing work New York, NY
- remote construction estimator New York, NY
- remote fashion New York, NY