Knowledge Retrieval QA Reviewer & Rubric Architect
AuraOne, Inc.
AuraOne, Inc. is seeking a Knowledge Retrieval Research QA Specialist as a remote contractor to review prompts and model outputs against AuraOne's quality rubric. You will label edge cases, write structured feedback, and help retrain the evaluation system. Responsibilities include comparing paired responses, labeling hallucinations and safety issues, and calibrating scores against gold standards. This role requires strong written reasoning and reliable async availability for at least 10 hours #J-18808-Ljbffr AuraOne, Inc.
$10 - $40 per hour
Review AI image edits against prompts, checking realism, defects, and... ..., or rework using a quality rubric. Write clear annotations and... ...ambiguous. Tools and setup: Working knowledge of Adobe Photoshop, plus a... ...annotation and rubric-based QA. Familiarity with Photoshop Generative...SuggestedHourly payContract workPart timeFor contractorsRemote workWorldwide$200k - $250k
...Nashville, Tennessee Job Description The Knowledge Architect is responsible for the technical... ...document processing pipelines, and reusable retrieval services. The role determines when to... ...platform provider. Diagnose issues, review vendor designs, manage escalations, and...SuggestedFull time- AuraOne, Inc. is seeking an AI Agent Interaction Specialist to remotely review ai agent interaction evaluation prompts and responses against AuraOne's quality rubric. You will compare paired outputs, label edge cases, and write structured feedback for the modeling team...SuggestedRemote work10 hours per week
- ...you will ownThe structure of our product knowledge Define and keep the content model for... ...into clear output that other people can review. Improve the way we generate this output... ...how AI assistants use content: how they retrieve it, why a well-written article can still...SuggestedWork at officeLocal areaRemote workVisa sponsorshipFlexible hours
$134k - $179k
...Learn more at .What you'll doThe Knowledge Architecture team at... ...role is a dedicated Knowledge Architect for Fleet Engineering, embedded... ..., Markdown, and pull request review.Experience with enterprise search... ...and metadata affect retrieval.Experience defining information...SuggestedPermanent employmentFull timeTemporary workCasual workWork at officeFlexible hours$134k - $179k
...2025. Learn more at .What you'll doThe Knowledge Architecture team at CoreWeave owns how... ...roleAs an IC3 (Practitioner) Knowledge Architect, you will add dedicated documentation and... ...including how structure and metadata affect retrieval.Experience defining information...Full timeTemporary workCasual workWork at officeFlexible hours- Benchmark Dataset Ontology QA Reviewer is a remote evaluation track for reviewing benchmark dataset ontology qa evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback...Hourly payFor contractorsRemote work10 hours per week
- AI Trainer Jobs is seeking a UX/UI Expert reviewer for remote work to evaluate AI outputs within ux/ui specialist operations workflows.... ...assess tone, escalation, and stakeholder fit using a structured rubric and document the next steps for model training. You must...Remote jobPart timeFor contractors10 hours per week
- ...is seeking an Office Operations & Logistics Manager for remote review work evaluating AI outputs in supply chain workflows. You will grade... ...contractor role requires direct supply chain experience, strong rubric application, and clear written reasoning; 10+ hours weekly async...Remote jobHourly payFor contractorsWork at office
- ...Trainer Jobs is seeking a Retail Specialist contractor to remotely review AI outputs across retail operations workflows. You will grade... ...step for model training. Work involves applying multi-page rubrics, providing clear reasoning that cites the policy or workflow used...Remote jobFor contractors10 hours per week
- AuraOne, Inc. seeks a Data & Charts Expert contractor to review AI outputs against data charts operator playbooks and firm policy. You... ...to ensure accurate data interpretation and policy adherence across multi-page rubrics and real-world #J-18808-Ljbffr AuraOne, Inc.Remote jobFor contractors10 hours per week
- AuraOne, Inc. is seeking a meticulous Malay Linguistic Experts reviewer to work remotely as a contractor. You will evaluate AI outputs... ...experience in Malay linguistic operations, comfort applying multi-page rubrics consistently, clear written reasoning citing the policy or...Remote jobFor contractors
- You will grade tone, escalation logic, and policy adherence on a structured rubric, flag risks, and help ensure quality and reliability in automated tasks. This is a contractor role with flexible hours, requiring at least 10 hours per week and US work authorization. #J...Remote jobFor contractors10 hours per weekFlexible hours
- ...facial data collection referrals specialist operations. You will review outputs, grade against workflows, and flag policy gaps,... ...requires reliable async availability and experience with multi-page rubrics. Compensation is hourly, with rate confirmed after interview. Remote...Remote jobHourly payFor contractorsFlexible hours
- ...Property Damage and Restoration Expert to operate as a remote reviewer for AI output in senior property damage and restoration workflows... ...tone, escalation logic, and stakeholder fit using structured rubrics, documenting decisions, and maintaining inter-rater calibration...Remote work
- ...States seeks a remote Business Document Expert (German Speaker) to evaluate German generalist prompts and responses against AuraOne's rubric. You will label edge cases, provide structured feedback for retraining, and compare outputs to select the stronger answer with...Hourly payFor contractorsRemote work
- AuraOne is seeking reviewers for its remote Logistics & Planning Professionals track. You will assess AI outputs against supply chain workflows... ...tone, escalation logic, and stakeholder fit using a structured rubric. The role emphasizes clear written reasoning, meticulous...Remote jobFor contractors
- AuraOne, Inc. seeks an Open Source Contributor (GitHub) specialist to review AI outputs against our contributor workflows, policies, and... ...grade tone, escalation, and stakeholder fit using a structured rubric, flag risks, and document next steps for training. Remote US-eligible...Remote jobFor contractors10 hours per week
- AI Trainer Jobs is seeking a remote independent contractor to review AI outputs for Google Suite operations, grading against policies and... ...tone, escalation logic, and stakeholder fit using a structured rubric and document actionable next steps for model training....Remote jobPart timeFor contractors
- AuraOne, Inc. seeks a remote Weapons Systems Operator reviewer to assess AI outputs within operator workflows and firm policies. You will... ...grade tone, escalation, and stakeholder fit using a structured rubric, and document actionable next steps for model training. Ideal...Remote job
- AuraOne, Inc. is seeking a Creative Brief Production Reviewer to operate remotely as an independent contractor in the United States. The... ...experience with creative brief production operations, strong rubric application skills, and clear written reasoning. #J-18808-Ljbffr...Remote jobFor contractors
- ...evaluate AI outputs across retail operations workflows. You will review, grade, and document next steps to guide the modeling team in... ...experience in retail trade operations on real teams, plus strong rubric-driven assessment and clear policy reasoning. Availability of at...Remote jobHourly payFor contractors
- ...seeking a Veterans Service Representative specialist operations reviewer to assess AI outputs for adherence to workflows, policies, and... .... The role emphasizes precise written reasoning and consistent rubric application, with remote, US-eligible work as a CONTRACTOR and...Remote jobFor contractors10 hours per week
- ...Jobs is seeking a contractor for a Sales/Marketing Consultation track conducted remotely. Review AI outputs, grade workflow, tone, escalation logic, and stakeholder fit against a rubric, and capture the right next steps to inform model training. Candidates should have...Remote jobFor contractorsFlexible hours
- ...Inc. is seeking a Paid Photo Contributor (Cousins) to remotely review AI outputs within paid photo contributor operations workflows.... ...strong attention to detail and the ability to apply multi-page rubrics consistently, with at least 10 hours per week #J-18808-Ljbffr...Remote jobContract workFor contractors10 hours per week
- AI Trainer Jobs is seeking a detail-oriented QA reviewer to assess AI-generated or edited video content for quality, pacing, transitions... ...where you flag issues at timestamps and rate outputs using a rubric. You will need at least 12 months of hands-on video production...
- ...Human Data is seeking a Records Manager specialist operations reviewer to evaluate AI outputs across workflows and playbooks. You will... ...tone, escalation logic, and stakeholder fit using a structured rubric and flag gaps with severity tags to guide model training. As a...Remote jobFor contractors10 hours per week
- ...Inc. is seeking a Remote smartwatch user specialist operations reviewer to evaluate AI outputs within real-world workflows. You will apply... ...with at least 10 hours per week, reviewing multi-page rubrics, flagging escalations, and ensuring consistency across batches....Remote job10 hours per week
- ...seeking a remote Humanitarian Aid Representative contractor to review AI outputs against operating workflows, playbooks, and firm policy... ...tone, escalation logic, and stakeholder fit using a structured rubric, and flag policy gaps with clear severity tags. You will also document...Remote jobFor contractors
- AuraOne, Inc. seeks a remote reviewer to assess Google Suite outputs across Sheets, Docs, and Slides against workflows and firm policy.... ...tone, escalation logic, and stakeholder fit using a structured rubric. The role requires clear written reasoning, meticulous attention...Remote jobFor contractors10 hours per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Knowledge Retrieval QA Reviewer & Rubric Architect. Be the first to apply!
- pharmaceutical quality assurance New York, NY
- quality assurance auditor New York, NY
- qa technician New York, NY
- qa tech New York, NY
- quality assurance monitor New York, NY
- quality assurance test manager New York, NY
- qa software testing New York, NY
- software quality assurance New York, NY
- software qa New York, NY
- work from home quality assurance New York, NY


