Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Knowledge Retrieval QA Reviewer & Rubric Architect

AuraOne, Inc.

AuraOne, Inc. is seeking a Knowledge Retrieval Research QA Specialist as a remote contractor to review prompts and model outputs against AuraOne's quality rubric. You will label edge cases, write structured feedback, and help retrain the evaluation system. Responsibilities include comparing paired responses, labeling hallucinations and safety issues, and calibrating scores against gold standards. This role requires strong written reasoning and reliable async availability for at least 10 hours #J-18808-Ljbffr AuraOne, Inc.

Vacancy posted 12 hours ago
Similar jobs that could be interesting for youBased on the Knowledge Retrieval QA Reviewer & Rubric Architect in New York, NY vacancy
  • $10 - $40 per hour

    Review AI image edits against prompts, checking realism, defects, and...  ..., or rework using a quality rubric. Write clear annotations and...  ...ambiguous. Tools and setup: Working knowledge of Adobe Photoshop, plus a...  ...annotation and rubric-based QA. Familiarity with Photoshop Generative... 
    Suggested
    Hourly pay
    Contract work
    Part time
    For contractors
    Remote work
    Worldwide

    OpenTrain AI, Inc.

    New York, NY
    3 days ago
  • $200k - $250k

     ...Nashville, Tennessee Job Description The Knowledge Architect is responsible for the technical...  ...document processing pipelines, and reusable retrieval services. The role determines when to...  ...platform provider. Diagnose issues, review vendor designs, manage escalations, and... 
    Suggested
    Full time

    Pillsbury Winthrop Shaw Pittman

    New York, NY
    4 days ago
  • AuraOne, Inc. is seeking an AI Agent Interaction Specialist to remotely review ai agent interaction evaluation prompts and responses against AuraOne's quality rubric. You will compare paired outputs, label edge cases, and write structured feedback for the modeling team... 
    Suggested
    Remote work
    10 hours per week

    AuraOne, Inc.

    New York, NY
    22 hours ago
  •  ...you will ownThe structure of our product knowledge Define and keep the content model for...  ...into clear output that other people can review. Improve the way we generate this output...  ...how AI assistants use content: how they retrieve it, why a well-written article can still... 
    Suggested
    Work at office
    Local area
    Remote work
    Visa sponsorship
    Flexible hours

    Docplanner

    New York, NY
    22 hours ago
  • $134k - $179k

     ...Learn more at .What you'll doThe Knowledge Architecture team at...  ...role is a dedicated Knowledge Architect for Fleet Engineering, embedded...  ..., Markdown, and pull request review.Experience with enterprise search...  ...and metadata affect retrieval.Experience defining information... 
    Suggested
    Permanent employment
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    New York, NY
    22 hours ago
  • $134k - $179k

     ...2025. Learn more at .What you'll doThe Knowledge Architecture team at CoreWeave owns how...  ...roleAs an IC3 (Practitioner) Knowledge Architect, you will add dedicated documentation and...  ...including how structure and metadata affect retrieval.Experience defining information... 
    Full time
    Temporary work
    Casual work
    Work at office
    Flexible hours

    CoreWeave

    New York, NY
    1 day ago
  • Benchmark Dataset Ontology QA Reviewer is a remote evaluation track for reviewing benchmark dataset ontology qa evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback... 
    Hourly pay
    For contractors
    Remote work
    10 hours per week

    AuraOne, Inc.

    New York, NY
    4 days ago
  • AI Trainer Jobs is seeking a UX/UI Expert reviewer for remote work to evaluate AI outputs within ux/ui specialist operations workflows....  ...assess tone, escalation, and stakeholder fit using a structured rubric and document the next steps for model training. You must... 
    Remote job
    Part time
    For contractors
    10 hours per week

    Auraone

    New York, NY
    1 day ago
  •  ...is seeking an Office Operations & Logistics Manager for remote review work evaluating AI outputs in supply chain workflows. You will grade...  ...contractor role requires direct supply chain experience, strong rubric application, and clear written reasoning; 10+ hours weekly async... 
    Remote job
    Hourly pay
    For contractors
    Work at office

    AuraOne

    New York, NY
    3 days ago
  •  ...Trainer Jobs is seeking a Retail Specialist contractor to remotely review AI outputs across retail operations workflows. You will grade...  ...step for model training. Work involves applying multi-page rubrics, providing clear reasoning that cites the policy or workflow used... 
    Remote job
    For contractors
    10 hours per week

    Auraone

    New York, NY
    3 days ago
  • AuraOne, Inc. seeks a Data & Charts Expert contractor to review AI outputs against data charts operator playbooks and firm policy. You...  ...to ensure accurate data interpretation and policy adherence across multi-page rubrics and real-world #J-18808-Ljbffr AuraOne, Inc.
    Remote job
    For contractors
    10 hours per week

    AuraOne, Inc.

    New York, NY
    12 hours ago
  • AuraOne, Inc. is seeking a meticulous Malay Linguistic Experts reviewer to work remotely as a contractor. You will evaluate AI outputs...  ...experience in Malay linguistic operations, comfort applying multi-page rubrics consistently, clear written reasoning citing the policy or... 
    Remote job
    For contractors

    AuraOne, Inc.

    New York, NY
    22 hours ago
  • You will grade tone, escalation logic, and policy adherence on a structured rubric, flag risks, and help ensure quality and reliability in automated tasks. This is a contractor role with flexible hours, requiring at least 10 hours per week and US work authorization. #J... 
    Remote job
    For contractors
    10 hours per week
    Flexible hours

    AuraOne, Inc.

    New York, NY
    4 days ago
  •  ...facial data collection referrals specialist operations. You will review outputs, grade against workflows, and flag policy gaps,...  ...requires reliable async availability and experience with multi-page rubrics. Compensation is hourly, with rate confirmed after interview. Remote... 
    Remote job
    Hourly pay
    For contractors
    Flexible hours

    Auraone

    New York, NY
    12 hours ago
  •  ...Property Damage and Restoration Expert to operate as a remote reviewer for AI output in senior property damage and restoration workflows...  ...tone, escalation logic, and stakeholder fit using structured rubrics, documenting decisions, and maintaining inter-rater calibration... 
    Remote work

    AuraOne Human Data

    New York, NY
    4 days ago
  •  ...States seeks a remote Business Document Expert (German Speaker) to evaluate German generalist prompts and responses against AuraOne's rubric. You will label edge cases, provide structured feedback for retraining, and compare outputs to select the stronger answer with... 
    Hourly pay
    For contractors
    Remote work

    Auraone

    New York, NY
    1 day ago
  • AuraOne is seeking reviewers for its remote Logistics & Planning Professionals track. You will assess AI outputs against supply chain workflows...  ...tone, escalation logic, and stakeholder fit using a structured rubric. The role emphasizes clear written reasoning, meticulous... 
    Remote job
    For contractors

    Auraone

    New York, NY
    4 days ago
  • AuraOne, Inc. seeks an Open Source Contributor (GitHub) specialist to review AI outputs against our contributor workflows, policies, and...  ...grade tone, escalation, and stakeholder fit using a structured rubric, flag risks, and document next steps for training. Remote US-eligible... 
    Remote job
    For contractors
    10 hours per week

    AuraOne, Inc.

    New York, NY
    4 days ago
  • AI Trainer Jobs is seeking a remote independent contractor to review AI outputs for Google Suite operations, grading against policies and...  ...tone, escalation logic, and stakeholder fit using a structured rubric and document actionable next steps for model training.... 
    Remote job
    Part time
    For contractors

    Auraone

    New York, NY
    22 hours ago
  • AuraOne, Inc. seeks a remote Weapons Systems Operator reviewer to assess AI outputs within operator workflows and firm policies. You will...  ...grade tone, escalation, and stakeholder fit using a structured rubric, and document actionable next steps for model training. Ideal... 
    Remote job

    AuraOne, Inc.

    New York, NY
    2 days ago
  • AuraOne, Inc. is seeking a Creative Brief Production Reviewer to operate remotely as an independent contractor in the United States. The...  ...experience with creative brief production operations, strong rubric application skills, and clear written reasoning. #J-18808-Ljbffr... 
    Remote job
    For contractors

    AuraOne, Inc.

    New York, NY
    4 days ago
  •  ...evaluate AI outputs across retail operations workflows. You will review, grade, and document next steps to guide the modeling team in...  ...experience in retail trade operations on real teams, plus strong rubric-driven assessment and clear policy reasoning. Availability of at... 
    Remote job
    Hourly pay
    For contractors

    AuraOne, Inc.

    New York, NY
    2 days ago
  •  ...seeking a Veterans Service Representative specialist operations reviewer to assess AI outputs for adherence to workflows, policies, and...  .... The role emphasizes precise written reasoning and consistent rubric application, with remote, US-eligible work as a CONTRACTOR and... 
    Remote job
    For contractors
    10 hours per week

    AuraOne Human Data

    New York, NY
    4 days ago
  •  ...Jobs is seeking a contractor for a Sales/Marketing Consultation track conducted remotely. Review AI outputs, grade workflow, tone, escalation logic, and stakeholder fit against a rubric, and capture the right next steps to inform model training. Candidates should have... 
    Remote job
    For contractors
    Flexible hours

    Auraone

    New York, NY
    3 days ago
  •  ...Inc. is seeking a Paid Photo Contributor (Cousins) to remotely review AI outputs within paid photo contributor operations workflows....  ...strong attention to detail and the ability to apply multi-page rubrics consistently, with at least 10 hours per week #J-18808-Ljbffr... 
    Remote job
    Contract work
    For contractors
    10 hours per week

    AuraOne Human Data

    New York, NY
    12 hours ago
  • AI Trainer Jobs is seeking a detail-oriented QA reviewer to assess AI-generated or edited video content for quality, pacing, transitions...  ...where you flag issues at timestamps and rate outputs using a rubric. You will need at least 12 months of hands-on video production... 

    Claru

    New York, NY
    3 days ago
  •  ...Human Data is seeking a Records Manager specialist operations reviewer to evaluate AI outputs across workflows and playbooks. You will...  ...tone, escalation logic, and stakeholder fit using a structured rubric and flag gaps with severity tags to guide model training. As a... 
    Remote job
    For contractors
    10 hours per week

    AuraOne Human Data

    New York, NY
    1 day ago
  •  ...Inc. is seeking a Remote smartwatch user specialist operations reviewer to evaluate AI outputs within real-world workflows. You will apply...  ...with at least 10 hours per week, reviewing multi-page rubrics, flagging escalations, and ensuring consistency across batches.... 
    Remote job
    10 hours per week

    AuraOne, Inc.

    New York, NY
    4 days ago
  •  ...seeking a remote Humanitarian Aid Representative contractor to review AI outputs against operating workflows, playbooks, and firm policy...  ...tone, escalation logic, and stakeholder fit using a structured rubric, and flag policy gaps with clear severity tags. You will also document... 
    Remote job
    For contractors

    AuraOne, Inc.

    New York, NY
    2 days ago
  • AuraOne, Inc. seeks a remote reviewer to assess Google Suite outputs across Sheets, Docs, and Slides against workflows and firm policy....  ...tone, escalation logic, and stakeholder fit using a structured rubric. The role requires clear written reasoning, meticulous attention... 
    Remote job
    For contractors
    10 hours per week

    AuraOne, Inc.

    New York, NY
    12 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Knowledge Retrieval QA Reviewer & Rubric Architect. Be the first to apply!