Remote Retrieval Relevance Evaluator and Rubric Specialist
AuraOne
New York, NY
- Remote job
AuraOne is seeking a remote Retrieval Relevance Task Evaluator to review prompts and responses against our quality rubric. You will compare outputs, assign severity tags, and provide structured feedback for model retraining. This contractor role requires careful reasoning, attention to detail, and ability to work asynchronously for at least 10 hours per week. Experience with rubrics and annotation is ideal. You’ll help label edge cases and support rubric updates. #J-18808-Ljbffr AuraOne
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Remote Retrieval Relevance Evaluator and Rubric Specialist in New York, NY vacancy
- ...Retrieval Relevance Task Evaluator is a remote evaluation track for reviewing retrieval relevance task evaluation... ...responses against AuraOne's quality rubric. Reviewers compare paired outputs,... ...US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR...Remote jobHourly payFor contractors10 hours per week
- AuraOne is seeking a Prompt Robustness Model Evaluation Specialist to remotely review prompts and model outputs, applying AuraOne's quality rubric. You will compare paired responses, label edge cases, and provide structured feedback for retraining. This contractor role...Remote jobFor contractors10 hours per week
$80 - $120 per hour
...Process improvement / SOPs Evaluator Type: Contract Compensation... ...80–$120/hour Location: Remote Role Responsibilities... ...against domain-specific quality rubrics. Identify factual,... ...Must-Have ~5+ years of relevant professional experience in Process...Remote workContract workSummer workWork at officeMercor
Houston, TXa month ago- AuraOne seeks a remote contractor to evaluate multi-turn ranking reward model evaluation prompts and responses. You will compare outputs, assign severity... ...structured feedback to retrain the model using AuraOne's rubric. You will identify hallucinations, instruction-following...Remote jobFor contractors
$80 - $120 per hour
...Humanities / arts / culture Evaluator Type: Contract Compensation... ...80–$120/hour Location: Remote Role Responsibilities... ...domain-specific quality rubrics. Identify factual, aesthetic... ...Must-Have ~5+ years of relevant professional experience in Humanities...Remote workContract workSummer workWork at office$80 - $120 per hour
...philanthropy / community programs Evaluator Type: Contract... ...80–$120/hour Location: Remote Role Responsibilities Evaluate... ...domain-specific quality rubrics. Identify factual, aesthetic... ...Must-Have ~5+ years of relevant professional experience in Nonprofit...Remote workContract workSummer workWork at office- AuraOne is seeking a Music Audio Expert - Telugu for remote evaluation of audio annotation prompts and responses, ensuring alignment with quality rubrics. This contractor role involves comparing outputs, assigning severity, and providing structured feedback for model retraining...Remote jobFor contractors
$15 per hour
Telus Digital is seeking a part-time remote freelancer to review online advertisements and rate their relevance to search terms. Your role requires excellent English and Dutch, with no prior experience needed beyond passing a basic assessment. This long-term project includes...Remote jobHourly payPart timeFreelanceFlexible hours- ...provider is seeking a Personalized Ads Evaluator for a part-time position. This entry-level... ...online advertisements to assess their relevance to search terms, providing detailed feedback... .... The position allows up to 20 hours of remote work weekly and requires passing a basic...Remote jobPart time
- AuraOne is seeking a Retrieval Relevance Research QA Specialist to remotely evaluate model outputs against a versioned rubric and provide structured feedback. You will compare responses, label issues, and suggest rubric improvements as part of contractor work for AuraOne...Remote jobFor contractors
- AuraOne is seeking a Brand Design Quality Evaluator to remotely assess design quality prompts and model outputs. You will compare paired responses, apply a versioned rubric, and provide structured feedback for retraining the modeling team. Responsibilities include labeling...Remote jobFor contractors
- AuraOne seeks a remote Bambara Language Expert contractor to review prompts and model outputs, apply a versioned rubric, and provide structured feedback for retraining. You will compare paired responses, label edge cases, and document recurring failure modes for continuous...Remote jobFor contractors10 hours per week
- Datavant is seeking a part-time Chart Retrieval Specialist in the Miami area. You will visit provider offices to retrieve medical records, scan... ...charts, and maintain HIPAA-compliant communications with a remote team. The role offers hybrid work (home office and field),...Remote workPart timeWork from homeHome office
- Pearson is seeking a Scoring Assistant for School Administrator and Superintendent credential evaluation. The role is remote and part-time, with sessions typically lasting 2-3 days monthly and scheduled between 8:00 a.m. and 4:00 p.m. PST. Training and review are provided...Remote jobPart timeFlexible hours
- ...Datavant team for a specialized, part‑time role as a Chart Retrieval Specialist. This role involves visiting various provider offices to retrieve... ...records, maintaining consistent communication with your remote team, providing daily updates regarding progress, challenges...Remote workHourly payPart timeCasual workHome officeMonday to Friday3 days per week
Datavant2
Louisville, KY5 days ago - ...Datavant team for a specialized, part-time role as a Chart Retrieval Specialist. This role involves visiting various provider offices to retrieve... ...records, maintaining consistent communication with your remote team, providing daily updates regarding progress, challenges...Remote workHourly payPart timeCasual workHome officeMonday to Friday3 days per week
Datavant
Pittsburgh, PA1 day ago $80 - $120 per hour
...Management / Implementation Planning Evaluator $80-$120 / hr Hourly contract Remote About the role We are... ...Requirements (must have) ~5+ years of relevant professional experience in... ...against domain-specific quality rubrics. Identify factual, aesthetic,...Remote workHourly payContract workFor contractorsWork at officeDorado
United States4 days ago$80 - $120 per hour
...Management / Implementation Planning Evaluator This position is listed on... ...communication. Working remotely and asynchronously, you will... ...against established quality rubrics and domain-specific standards... ...Collaborate asynchronously with relevant stakeholders and work...Remote workHourly payContract workWork at officeFlexible hoursJobgether
United States1 day ago- Datavant is hiring a specialized, part-time Chart Retrieval Specialist. This role involves visiting provider offices to retrieve medical records, scanning or uploading charts, and coordinating with a remote team to ensure accuracy and deadlines. Candidates must have a...Remote workPart timeMonday to FridayFlexible hours
- A leading localization company is seeking a Slovak Ads Analyst to help improve AI advertising relevance. This remote, flexible role allows you to work 5 to 20 hours per week with no interviews required. Ideal candidates are fluent in Slovak, knowledgeable about modern...Remote jobFreelanceFlexible hours
- AuraOne is seeking a remote Business Document Expert (Portuguese Speaker) to review prompts and responses against our quality rubric. You will evaluate Portuguese generalist outputs, label edge cases, and provide structured feedback to retrain models. This is a contractor...Remote jobFor contractorsFlexible hours
- AuraOne is seeking a Tagalog Language Expert to review generalist evaluation prompts and responses against the company’s quality rubric. This is a remote contractor role focusing on structuring feedback, labeling edge cases, and generating regression data for retraining...Remote jobFor contractors
$14 per hour
...achieve optimal claim outcomes. We are looking for a Record Retrieval Specialist to join our Medical Records team. Job Description As a Record... ...‑task Detail‑oriented Ability to work independently in a remote setting Knowledge of basic Microsoft Office programs (preferred...Remote workHourly payFull timeWork at officeMonday to FridayShift work$39.16 - $60.42 per hour
...Description: The CV Invasive Specialist I performs duties necessary... ...ARRT-VI only or RN (previous relevant RN experience required)... ...patients. Will also bend to retrieve, lift, and carry supplies and... ...expectations. If applying for a remote or hybrid role, this includes...Remote workHourly payFull timeRelocation packageShift workIntermountain Health
Brighton, CO14 hours ago- AuraOne is seeking a remote News Monitoring Task Evaluator to assess evaluation prompts and responses against our quality rubric. You will compare paired outputs, write rationales, and label edge cases to aid model retraining and quality control. Join as a contractor working...Remote jobFor contractors
- AuraOne is seeking a remote Rubric Calibration Model Evaluation Specialist contractor in the United States. The role focuses on evaluating rubric calibration model outputs, labeling issues, and providing structured feedback to retrain the model. You will compare paired...Remote jobFor contractors
$80 - $120 per hour
...Pricing / ROI / revenue economics Evaluator Type: Contract... ...80–$120/hour Location: Remote Role Responsibilities Evaluate... ...domain-specific quality rubrics. Identify factual, aesthetic... ...ensure consistency and domain relevance. Work independently and...Remote workContract workSummer workWork at officeMercor
San Francisco, CA7 days ago$80 - $120 per hour
...contracts / diligence / redlines Evaluator to assess AI-generated... ...This role requires 5+ years of relevant experience and native or professional... ...will evaluate quality rubrics, provide structured feedback,... ...is $80-$120/hour with a fully remote work environment. #J-18808-...Remote job- ...Job Description Job Description Records Retrieval Specialist I Employment Type: Full-Time Non-Exempt Department: Retrieval Schedule... ...Working Conditions • Primarily office-based or remote work environment. • Frequent use of phone, computer, and...Remote workFull timeWork at officeMonday to Friday
Retrev LLC
Pensacola, FL17 days ago $30 per hour
...Compensation: $20-$30/hour Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language... ...agent systems using defined rubrics and quality standards. Review multi... ...and communicate insights to relevant stakeholders. Requirements Strong...Remote jobHourly payContract work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote Retrieval Relevance Evaluator and Rubric Specialist. Be the first to apply!
Related searches
- program evaluator New York, NY
- work from home web search evaluator New York, NY
- clinical evaluator New York, NY
- evaluator New York, NY
- ai evaluator New York, NY
- work from home social media evaluator New York, NY
- quality evaluator New York, NY
- education evaluator New York, NY
- social media evaluator New York, NY
- ads evaluator New York, NY