Competitive Research Task Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Competitive Research Task Evaluator is a remote review track for evaluating AI outputs across competitive research task research review reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the correct method so the modeling team can train on it.
Why this role matters
Competitive Research Task research review models live or die on whether their derivations actually hold up under scrutiny. AuraOne uses scientific specialists to grade outputs the way a peer reviewer would — checking assumptions, reproducing key steps, and capturing the right method alongside the wrong one.
Responsibilities
- Review AI outputs against current competitive research task research review methods, conventions, and prior work for Competitive Research Task Evaluator assignments.
- Reproduce or sanity-check key derivations, calculations, or experimental claims.
- Flag dimensional, methodological, and citation errors with structured severity tags.
- Capture the corrected reasoning or worked example so the modeling team can train on it.
- Adjudicate disputed answers against textbooks, papers, or community standards.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Graduate-level training or equivalent applied experience in competitive research task research review or a closely related field for Competitive Research Task Evaluator work.
- Hands-on experience publishing, teaching, or advising on the topic at a professional level.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that cites methods, papers, or worked examples.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Reproduce a competitive research task research review derivation from a model output and flag any algebraic or dimensional errors.
- Grade a model's literature summary against the cited papers and rate the citation quality.
- Adjudicate a disputed answer between two reviewers using textbook methods.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- PhD, postdoc, or industry research experience in the topic area.
- Prior work reviewing AI-assisted research tooling and its failure modes.
- Multilingual fluency for non-English papers and corpora.
Skills
- Scientific reasoning
- Method validation
- Citation review
- Quantitative analysis
- Competitive Research Task research review
- Web research
- Source grounding
- Browser automation
- Competitive
- Research
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Competitive Research Task Evaluator [Remote] in Remote vacancy
$40 - $100 per hour
...countries English-language work About AI Training and Scientific Evaluation AI training is the human side of building modern... ...challenge models with realistic scenarios, calculations, and research-style reasoning tasks. Focus on scientific evaluation of AI outputs Apply...SuggestedHourly payContract workPart timeFor contractorsRemote work- ...Program Analytics & Insights Evaluator IIon the project, you will be... ...evaluation frameworks, logic models, research questions, and evaluation... ...and prioritize multiple tasks in a fast-paced and dynamic environmentStrong... ...a benefits package that is competitive for project model-focused...SuggestedLocal area
$16 - $20 per hour
...Research Evaluator Position The research team at Penn State Ross and Carol Nese College of Nursing is hiring part-time research evaluators for projects focused on dementia care in assisted living settings. The research evaluator will assist with in-person recruitment...SuggestedHourly payPart timeSummer workPenn State University
Harrisburg, PA1 day ago$24 per hour
...seeking an AI Trainer with advanced Tamil fluency to evaluate AI models' understanding of the Tamil language's... ...intelligence, and a suitable technical setup. Competitive compensation is offered at up to $24/hr for tasks, with flexible hours and remote work options. #J-...SuggestedRemote jobFlexible hours- ...by our partnership with EQT. Website: Linkedin Job Title: Evaluator - Political Science Location: Remote (USA) Job Type: Contract... ...from quality checks. Stay Current: Stay updated with the latest research, guidelines, and advancements in your area of expertise,...SuggestedContract workRemote workWorldwide
$22 per hour
...of technological advancement and maintain a competitive edge in their respective markets. Job Overview As a Content Quality Evaluator, English, you will assess the quality of... ...not required Ability to perform repetitive tasks without degradation in quality. Detail-oriented...Work at office- ...Robot Incident Replay Task Evaluator is a remote evaluation track for reviewing robot incident replay task evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback...Remote jobHourly payFor contractors10 hours per week
- ...Government Policy and Legislative Research Evaluator is a remote review track for evaluating AI outputs in policy review workflows. Reviewers... ...async availability for at least 10 hours per week. Example tasks Verify the citations in a model's policy review memo and...Remote jobHourly payFor contractors10 hours per week
- ...Biomedical Research AI Evaluator is a remote clinical-review track for evaluating AI outputs that touch biomedical. Reviewers grade differential... ...availability for at least 10 hours per week. Example tasks Grade a model's differential-diagnosis reasoning on a biomedical...Remote jobHourly payFor contractors10 hours per week
- ...Surgical Robotics Task Evaluator is a remote review track for evaluating robotics review outputs against real-world physical-AI constraints. Reviewers grade trajectory choices, perception assumptions, and safety envelopes; flag failure modes; and document the corrected...Remote jobHourly payFor contractors10 hours per week
- ...Equity Research AI Evaluator is a remote review track for evaluating AI outputs across finance and risk workflows. Reviewers grade calculations... ...availability for at least 10 hours per week. Example tasks Re-perform a finance and risk calculation produced by a model...Remote jobHourly payFor contractorsWork experience placement10 hours per week
- ...Dual-Use Research Risk Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios... ...availability for at least 10 hours per week. Example tasks Construct a 5-turn adversarial conversation that bypasses...Remote jobHourly payFor contractors10 hours per week
$80 - $120 per hour
Gridnaut Recruiting is hiring a remote Market research / competitive intelligence Evaluator contractor (pay $80-$120/hr). Contribute to frontier AI research and evaluation work. Ideal candidates: Remote contractor engagement supporting a leading AI lab.; Remote AI evaluation...Hourly payContract workFor contractorsRemote workFlexible hours$80 - $120 per hour
...About the role We are hiring expert Evaluators in Market research / competitive intelligence to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise...Hourly payContract workWork at officeRemote work- ...a project focused on advancing next-generation AI systems. Remote involvement, contractor role, and strong research leadership are essential. You will evaluate complex physics arguments, calibrate expert judgments, and draft rigorous written evaluations for senior researchers...Remote jobFor contractors
$30 per hour
...Chicago, IL to train AI models. You will complete AI tasks and assess AI performance in Dutch, with pay at... ...the flexibility of working from home with competitive rates and direct payment through PayPal. Pass the evaluation, and you can start within 15 minutes. #J-18808-...Remote jobWork from home- ...context to support a range of AI training and evaluation projects . In this role, you will work across diverse task types, including evaluating prompts and AI-... ...fairly. We respect your time and your expertise. Competitive rates, prompt payments, no chasing invoices....Extra incomeFull timeFor contractorsLocal area10 hours per week
Lilt
Remotea month ago - ...compassionate, expert care, and to support recovery through education and research. Joining our PAMily allows you to work in a collaborative... ...exposure to a variety of patient care levels. Aside from our competitive pay, generous paid benefit time, and excellent insurance...Full timeLocal area
$75k - $110k
...or recreational properties is a plus—but not required Benefits Competitive salary based on experience ($75,000 to $110,000) Medical insurance... ..., respected firm with over 70 years of industry experience A Task and Duty Guide for this position is available upon request. Pay...Temporary workRemote workForecon
New York, NY4 days ago$20 per hour
...is hiring a Digital Web Designer. In this remote role, you will evaluate AI-generated designs and provide feedback to enhance the model’... .... The position offers flexible hours and pays $20+ USD/hr for general projects and $40+ for design-focused tasks. #J-18808-LjbffrRemote workFlexible hoursDataAnnotation
United States1 day ago$31.2k - $33.28k
...next steps. Our partner is looking for a German Language Data Evaluator based in United States. This is a remote opportunity for a bilingual... ..., and overall project performance. Manage assigned tasks efficiently while maintaining accuracy and meeting established...Hourly payFull timeRemote workWork from homeMonday to FridayFlexible hours- ...Collects, analyzes, and reports on data to evaluate effectiveness of program services.Core... ...working with both quantitative and qualitative research methods. Proficiency with applicable... ...and self-direction to complete assigned tasks and identify tools and approaches for analysis...Full timeSummer workLocal area
Family Connections
East Orange, NJ2 days ago $30 per hour
...Over 35,000 AI developers, researchers, and organizations use Prolific... ...Experts for a high-level AI evaluation project. AI models are... ...and now tackle complex design tasks such as layout, typography,... ...minutes, and start enjoying competitive pay rates, flexible hours, and...Remote jobWork from homeFlexible hours$11.5 per hour
...A leading educational institution is offering a part-time remote position as an Online Task Contributor. In this role, you will evaluate and provide feedback on content to enhance search engine results and quality. No prior experience is needed, but proficiency in English...Hourly payPart timeRemote workUniversity of Delaware
United States1 day ago- A leading AI research accelerator is seeking a contractor for a role focused on evaluating North American teen humor through structured ratings and clear explanations.... ...appeal. This flexible remote position promises competitive pay and engagement in innovative AI projects...Remote jobContract workFor contractorsFreelanceFlexible hours
$50 - $60 per hour
...intelligence. Contributors review examples, test model behavior, evaluate responses, and identify errors so AI systems become more... ...hours per week Work directly with cutting-edge AI evaluation tasks Use practical analytics expertise to improve AI systems The Role...Hourly payPart timeFor contractorsRemote workWorldwideFlexible hours$45 - $85 per hour
Ixolabs is seeking an Animation Quality Evaluator to refine AI-generated animations. The ideal candidate will evaluate motion fluidity... ...narratives. The role is part-time (15-25 hours/week) and offers competitive compensation of $45 - $85/hr. Applicants should possess 5+...Remote jobPart time$50 per hour
...Specialists to help train AI models using your expertise. You'll evaluate AI-generated designs and ensure usability and accessibility... ...least one year of professional experience. This position offers competitive pay (up to $50/hour) and requires proficiency in design tools...Remote jobWork from homeFlexible hours- A talent marketplace is seeking soccer experts to evaluate live soccer games. The role involves assessing AI-generated commentary, scoring... ...to refine AI's soccer insights through expert evaluations. Competitive pay is offered, providing an opportunity for passionate soccer...Contract work
- A leading AI research accelerator is seeking a contractor to evaluate North American teen humor. The ideal candidate will review and rate humorous content popular... ...role offers flexible, project-based work with competitive pay and the chance to learn about advanced AI...Remote jobFor contractorsFreelanceFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Competitive Research Task Evaluator [Remote]. Be the first to apply!
