Statistics and Probability AI Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Statistics and Probability AI Evaluator is a remote review track for evaluating AI outputs across statistics reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the correct method so the modeling team can train on it.
Why this role matters
Statistics models live or die on whether their derivations actually hold up under scrutiny. AuraOne uses scientific specialists to grade outputs the way a peer reviewer would — checking assumptions, reproducing key steps, and capturing the right method alongside the wrong one.
Responsibilities
- Review AI outputs against current statistics methods, conventions, and prior work for Statistics and Probability AI Evaluator assignments.
- Reproduce or sanity-check key derivations, calculations, or experimental claims.
- Flag dimensional, methodological, and citation errors with structured severity tags.
- Capture the corrected reasoning or worked example so the modeling team can train on it.
- Adjudicate disputed answers against textbooks, papers, or community standards.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Graduate-level training or equivalent applied experience in statistics or a closely related field for Statistics and Probability AI Evaluator work.
- Hands-on experience publishing, teaching, or advising on the topic at a professional level.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that cites methods, papers, or worked examples.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Reproduce a statistics derivation from a model output and flag any algebraic or dimensional errors.
- Grade a model's literature summary against the cited papers and rate the citation quality.
- Adjudicate a disputed answer between two reviewers using textbook methods.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- PhD, postdoc, or industry research experience in the topic area.
- Prior work reviewing AI-assisted research tooling and its failure modes.
- Multilingual fluency for non-English papers and corpora.
Skills
- Scientific reasoning
- Method validation
- Citation review
- Quantitative analysis
- Statistics
- Science and advanced mathematics
- AI evaluation
- Rubric writing
- Expert review
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Statistics and Probability AI Evaluator [Remote] in Remote vacancy
$14.5 per hour
A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical...SuggestedHourly payPart timeRemote workFlexible hoursWelo Data
United States4 days ago$20 per hour
A tech company specializing in AI is hiring a Digital Web Designer. In this remote role, you will evaluate AI-generated designs and provide feedback to enhance the model’s understanding of aesthetics. An ideal candidate will have a strong background in UI/UX design and...SuggestedRemote workFlexible hoursDataAnnotation
United States4 days ago- ...Seeking a full-time Remote AI Research Evaluator with a PhD in Quantitative Finance to assess and enhance AI models' capabilities in financial reasoning and quantitative analysis through flexible, contract-based work. Key responsibilities Assessing the factuality and...SuggestedFull timeContract workRemote workFlexible hours
Virtual Vocations Inc
United States3 days ago $11.5 per hour
...position as an Online Task Contributor. In this role, you will evaluate and provide feedback on content to enhance search engine results... ...11.50 hourly, based on task completion, with a supportive community of contributors involved in AI advancements. #J-18808-Ljbffr...SuggestedHourly payPart timeRemote workUniversity of Delaware
United States4 days ago$20 per hour
A leading AI development company in the United States is seeking detail-oriented individuals for remote opportunities in training AI... ...include developing prompts, writing high-quality responses, and evaluating AI outputs. The ideal candidates are fluent in English with...SuggestedHourly payRemote workFlexible hoursSupportFinity
United States4 days ago- YO AI Labs is seeking an Investment & Finance Expert to work remotely as a contractor. The role focuses on evaluating AI outputs related to valuation, financial modeling, markets, and investments, and crafting expert prompts plus reference answers to improve model reasoning...Remote jobFor contractors
$20 - $30 per hour
A remote-focused technology firm is looking for an individual proficient in evaluating large language model outputs. The role involves assessing AI systems, reviewing workflows, and providing actionable feedback to enhance product quality. Ideal candidates will have strong...Remote jobHourly pay- YO AI Labs is seeking an AI Evaluation Specialist to remotely assess enterprise AI outputs against detailed rubrics, measuring accuracy, relevance and quality. You will provide concise, actionable feedback to drive improvements. Responsibilities include identifying reasoning...Remote jobFor contractors
- Obsidian is seeking expert Evaluators for a remote, hourly role focused on reviewing AI-generated work products for quality and accuracy. The successful candidates will leverage their subject matter expertise to assess various documents, spreadsheets, and slide decks....Remote jobHourly payWork at office
- About the role Hindi Localization AI Evaluator is a remote evaluation track for reviewing hindi generalist evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the...Hourly payFor contractorsRemote work10 hours per week
- Obsidian is seeking expert Evaluators in Humanities / arts / culture to review AI-generated work products for accuracy, rigor, and domain quality. You will apply subject-matter expertise to grade outputs and provide structured feedback. This is a remote, hourly engagement...Remote jobHourly payWork at office
- AI Trainer Jobs is seeking licensed clinicians to remotely evaluate AI outputs in internal medicine clinical reviews. You will assess differential diagnoses, dosing logic, and guideline adherence, and document corrected reasoning for model training. Ideal candidates hold...Remote jobHourly pay10 hours per week
- Feedinkoo is seeking a Graphic Designer to help train and improve AI models focused on UI/UX design, visuals, and user experiences. You will critique AI outputs, provide feedback, and guide model improvements to better reflect aesthetics and usability for designers. The...Remote work
- Turing is a leading AI company enabling the rapid deployment of advanced AI systems. We are seeking a contractor to evaluate chatbot responses across real-world small business scenarios, crafting prompts, and comparing outputs. This role emphasizes structured feedback,...Remote jobFor contractorsFreelance
- AuraOne is seeking a Chemicals Safety Risk Evaluator in a remote, US-eligible red-team track to stress-test AI systems against adversarial prompts. Reviewers craft attack scenarios, document failures, and tie each successful jailbreak to the violated policy clause so the...Remote jobFor contractors
$30 per hour
...Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous agent systems... ...stakeholders. Requirements Strong experience in LLM evaluation, AI output analysis, QA/testing, UX research, or similar analytical...Remote jobHourly payContract work- Mercor is seeking expert Evaluators in Clinical / biomedical / pharma to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. This is a remote, hourly engagement. You will apply deep subject-matter...Remote jobHourly pay
- OpenTrain AI, Inc. seeks a senior dermatology reviewer to provide expert clinical judgment on complex dermatology cases and evaluate longitudinal data for AI systems. You will document commentary according to guidelines and participate in consensus processes, with occasional...Remote jobPart timeFor contractors
- About the role JavaScript and TypeScript AI Evaluator is a remote evaluation track for reviewing javascript and typescript ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured...Hourly payFor contractorsRemote work10 hours per week
- Archangel Health AI is seeking Clinical AI Evaluators to review AI-generated clinical outputs, benchmark diagnostic reasoning, and refine responses to real-world medical queries. You will perform clinical accuracy auditing, RLHF ranking, error and harm identification,...Remote workFlexible hoursShift work
- YO AI Labs is seeking Content Writers & Editors to contribute to a customer project focused on text quality and AI evaluation. You will apply your writing and editing expertise to help train and evaluate next-generation AI systems through high-quality input. No prior AI...Remote job
- AI Trainer Jobs seeks a remote independent contractor to review AI outputs for fintech operations evaluation. You will assess workflow adherence, tone, and escalation logic, assigning severity tags and documenting next steps for model training. The role requires experience...Remote jobHourly payFor contractorsFlexible hours
- Prolific is seeking fluent Norwegian speakers to act as evaluators for AI training, performing side-by-side assessments of text and voice snippets to judge naturalness and authenticity. You will listen to audio clips and rate how naturally the AI speaks, providing detailed...Remote jobWork from homeFlexible hours
- A leading AI research collaboration in Atlanta is seeking a specialist to develop advanced physics problems and rigorously evaluate AI-generated solutions. Applicants must have a PhD or be near completion in Applied Physics or a related field, showcasing mastery in core...Remote jobFlexible hours
$24 per hour
Prolific is seeking Fluent Tamil speakers to serve as AI evaluators on a remote, freelance basis. You will assess how well AI models capture emotions and cultural nuance in Tamil, with pay up to $24/hr for one-hour tasks, and the option for shorter work sessions. Requirements...Remote jobFreelance- AI Trainer Jobs is seeking a Grocery Shopper Expert contractor for remote work in the US. You will review AI outputs across grocery shopper operations, grade workflow correctness, policy adherence, and stakeholder fit, and document next steps for model training. Experience...Remote jobFor contractors10 hours per week
- Obsidian is hiring expert Evaluators in Healthcare operations for a remote, hourly engagement. You will review AI-generated work products for accuracy, rigor, and domain quality, leveraging your extensive subject-matter expertise in the field. The ideal candidate has over...Remote jobHourly payWork at office
$65 - $70 per hour
Trust and Safety Policy AI Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team...Hourly payFor contractorsRemote work10 hours per week- About the role PhD Mathematics AI Evaluator is a remote review track for evaluating AI outputs across mathematics reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the correct method so the...Hourly payFor contractorsRemote work10 hours per week
- AI Trainer Jobs is seeking a Bilingual AI Response Evaluator for a remote, contract-based role. You will review bilingual prompts and responses, compare outputs, and provide structured feedback that guides model retraining. This position emphasizes careful judgment, clear...Remote jobHourly payContract work10 hours per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Statistics and Probability AI Evaluator [Remote]. Be the first to apply!
