Remote Helpfulness Ranking Evaluator & Rubric Assessor
AuraOne
New York, NY
- Remote job
AuraOne seeks a remote Helpfulness Ranking Reward Model Evaluator to review evaluation prompts and responses under a versioned rubric. You will compare paired outputs, assign severity, and provide structured feedback the modeling team can use for retraining. The role is contractor-based and requires US work authorization, with hourly compensation after interview. Responsibilities include labeling edge cases, auditing prompts, and maintaining reviewer-quality scores through weekly calibration #J-18808-Ljbffr AuraOne
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Remote Helpfulness Ranking Evaluator & Rubric Assessor in New York, NY vacancy
- ...Helpfulness Ranking Reward Model Evaluator is a remote evaluation track for reviewing helpfulness ranking reward model evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured...Remote jobHourly payFor contractors10 hours per week
- AuraOne seeks a remote contractor to evaluate multi-turn ranking reward model evaluation prompts and responses. You will compare outputs, assign severity tags... ...feedback to retrain the model using AuraOne's rubric. You will identify hallucinations, instruction-following...Remote jobFor contractors
- AuraOne is seeking a remote Retrieval Relevance Task Evaluator to review prompts and responses against our quality rubric. You will compare outputs, assign severity tags, and provide... ...rubrics and annotation is ideal. You’ll help label edge cases and support rubric updates...Remote jobFor contractors10 hours per week
- AuraOne is seeking a remote Video Understanding AI Evaluator contractor to review model outputs, apply a structured rubric, and provide rationale. You will compare responses, tag issues, and help refine prompts to improve reliability across batches. The role emphasizes...Remote jobFor contractors
- AuraOne is seeking a remote Pairwise Preference Reward Model Evaluator to review prompts and responses against our quality rubric. You will compare paired outputs, label edge cases, and provide structured feedback to help retrain the modeling team. This independent contractor...Remote jobHourly payFor contractors10 hours per week
- The College of American Pathologists (CAP) seeks a Technical Assessor to support accreditation for CAP 15189 laboratories. You will participate in onsite/remote assessments, review documentation, and help ensure conformance to ISO 15189 and related standards. Travel 50-...Remote work
- AuraOne is seeking a remote Spreadsheet QA / workbook maintenance Evaluator to help ensure high-quality AI outputs. You will review model responses, apply a versioned rubric, and provide structured feedback to retrain the system. As a contractor based in the US, you will...Remote jobFor contractors
- ...goal is to build a strong team of valuation professionals who will help shape the future of the company. We've built trusted... ...per-report compensation. Appraisers begin with a 90-day period to evaluate performance, reliability, and alignment with company standards....Remote workFor contractors
Banks Valuation
United States3 days ago $60k - $100k
...goal is to build a strong team of valuation professionals who will help shape the future of the company. We've built trusted... ...per-report compensation. Appraisers begin with a 90-day period to evaluate performance, reliability, and alignment with company standards....Remote workFor contractorsBanks Valuation
United States4 days ago$60k - $100k
...Job Location (Remote - Utah ) We are seeking Certified Residential Appraisers across... ...team of valuation professionals who will help shape the future of the company. We’ve... ...Appraisers begin with a 90-day period to evaluate performance, reliability, and alignment with...Remote workFor contractorsImmediate startBanks Valuation
United States3 days ago- ...appraisals. Join us to create best-in-industry appraisal reports and help shape the next generation of real estate appraisals. True... ...icy terrain. Field appraisers work outdoors at construction and remote sites in all seasons and weather, including inclement conditions...Remote workFull timeTraineeshipLong distance
True Footage
United States4 days ago - AuraOne is seeking a Capability Generalization Model Evaluation Specialist for a remote contractor role. You will evaluate model outputs against AuraOne's quality rubric, compare paired responses, and label edge cases with structured tags to guide retraining. Responsibilities...Remote workFor contractors
- AuraOne is seeking a Remote Receipt and Invoice Understanding Model Evaluator to review outputs against rubrics and provide structured feedback. You will compare paired responses, label edge cases, and contribute to retraining with rationale. Ideal candidates have prior...Remote jobFor contractors10 hours per week
- ...Property Appraiser I or II - Assessor’s Office - Goldendale, WA - Full... ...market analysis and to evaluate property in the surrounding area... ...leadership. A team player who helps the organization meet its... ...snow. It requires operating in remote areas of the county that do not...Remote workFull timeWork experience placementWork at officeLocal area
- ...stewardship and customer service consistently ranking as an industry leader in customer service... ..., experiences, and backgrounds to help SRP deliver on its mission of providing reliable... ...employees whose jobs can be performed remotely, and who have sufficient technical...Remote workFull timeTemporary workH1bLocal areaVisa sponsorshipWork visa3 days per week
- ...looking for Search Quality Evaluators to assess AI-generated... ...and recommendations - helping ensure that the AI... ...useful. This is a fully remote, flexible contract role... ...structured evaluation rubrics Compare multiple results side by side and rank them by quality Provide...Remote workHourly payOngoing contractContract workFreelanceFlexible hours
- AuraOne is seeking a remote Humanoid Task Task Evaluator to review and label humanoid task evaluation prompts and responses. You will compare paired outputs, apply a versioned rubric, and produce structured feedback for retraining the model. This contractor role requires...Remote jobFor contractors
- AuraOne is seeking a Remote Social Engineering Security Evaluator to review prompts and responses against our quality rubric, producing auditable labels, rationales, and regression cases. This role operates as an independent contractor to support the security evaluation...Remote jobFor contractors
- AuraOne is seeking an AI Data Annotation Expert for a remote contractor role focused on evaluating AI data annotation outputs against a versioned rubric. You will compare paired responses, label edge cases, and provide structured feedback to retrain the model. Responsibilities...Remote jobFor contractors
- ...JPMorgan Chase. As part of Risk Management and Compliance, you are at the center of keeping JPMorgan Chase strong and resilient. You help the firm grow its business in a responsible way by anticipating new and emerging risks, and using your expert judgement to solve...Remote workWork at officeLocal area
Chase
United States1 day ago - AuraOne is seeking a Prompt Robustness Model Evaluation Specialist to remotely review prompts and model outputs, applying AuraOne's quality rubric. You will compare paired responses, label edge cases, and provide structured feedback for retraining. This contractor role...Remote jobFor contractors10 hours per week
- AuraOne is seeking a remote Capability Elicitation Risk Evaluator to review prompts and model outputs. You will apply a versioned rubric to label edge cases, assign severity, and provide structured feedback for retraining. You will compare paired responses, select the stronger...Remote job
- ...Project Manager - Environmental & Engineering Property Condition Assessor for the Architectural, Engineering Construction Services (AEC... ...mannerPerform other consulting project duties as assignedWork from a remote-home setting with organizational support Frequent regional...Remote workFull timeFlexible hours
- AuraOne is seeking a Brand Design Quality Evaluator to remotely assess design quality prompts and model outputs. You will compare paired responses, apply a versioned rubric, and provide structured feedback for retraining the modeling team. Responsibilities include labeling...Remote jobFor contractors
- ...seeking an AI data reviewer to assess Welsh language evaluation outputs against a versioned rubric, tagging severity and providing rationales. You will compare... ...capture ambiguous prompts for rubric updates. This remote contractor role requires thorough attention to detail,...Remote jobFor contractors10 hours per week
- AuraOne seeks a remote Bambara Language Expert contractor to review prompts and model outputs, apply a versioned rubric, and provide structured feedback for retraining. You will compare paired responses, label edge cases, and document recurring failure modes for continuous...Remote jobFor contractors10 hours per week
- AuraOne is seeking an Educational Safety AI Evaluator for a remote, contractor-based role. You will review educational safety AI outputs, apply the quality rubric, and produce auditable labels, rationales, and regression cases for AuraOne Human Data. Responsibilities include...Remote jobFor contractors
$120k - $150k
Visa sponsorship provided: No Location: West Perth, WA Remote Status: no remote Pay Type: Annual Salary Salary: 120000-150000/Australian Dollars Occupational Categories: Real Estate/ Leasing/ Property Management Industry(ies): Real Estate/ Mortgage Are you a Licensed Valuer...Remote workLocal areaVisa sponsorshipFree visa- AuraOne is seeking a Czech Bilingual Expert for a remote contractor role to evaluate and label Czech bilingual outputs using AuraOne's rubric. The reviewer will compare responses, assign severity tags, and provide structured feedback to support model retraining. Responsibilities...Remote jobFor contractors
$12.65k
...professional assumptions and limiting conditions. * Analyze and evaluate appraisal information to determine final value of properties. *... ....**Circumstances involving tax status issues will be reviewed REMOTE/TELEWORK: This position may be eligible to work from an...Remote workFull timeWork experience placementWork at officeFlexible hoursState of Montana
Ennis, MT2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote Helpfulness Ranking Evaluator & Rubric Assessor. Be the first to apply!
Related searches
- program evaluator New York, NY
- work from home web search evaluator New York, NY
- clinical evaluator New York, NY
- evaluator New York, NY
- ai evaluator New York, NY
- work from home social media evaluator New York, NY
- quality evaluator New York, NY
- education evaluator New York, NY
- social media evaluator New York, NY
- security control assessor New York, NY