Regulatory Compliance AI Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Regulatory Compliance AI Evaluator is a remote review track for evaluating AI outputs in compliance workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can train on it.
Why this role matters
Compliance content is judged on precision. AuraOne uses qualified legal and policy reviewers to grade AI outputs the way a senior associate would — with citations, severity tags, and the corrected reasoning alongside the original.
Responsibilities
- Review AI outputs against current compliance statutes, case law, or policy for Regulatory Compliance AI Evaluator assignments.
- Verify citations, jurisdictions, and dates against authoritative sources.
- Flag risky reasoning, missed exceptions, and policy-adherence gaps with severity tags.
- Capture the corrected analysis or memo paragraph so the modeling team can train on it.
- Adjudicate disputed treatments against published authority or firm policy.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Active or recent qualification in compliance or an adjacent practice for Regulatory Compliance AI Evaluator work.
- Hands-on experience drafting, reviewing, or advising on real matters.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that cites authority and names the rule being applied.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Verify the citations in a model's compliance memo and flag any miscites with severity.
- Grade a model's statutory reasoning on a complex fact pattern and rate the analysis.
- Adjudicate a disputed treatment between two reviewers using current authority.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Bar admission or legal-equivalent credential in a relevant jurisdiction.
- Prior experience reviewing AI-assisted legal-research or compliance tooling.
- Bilingual experience for cross-jurisdiction matters.
Skills
- Legal research
- Citation review
- Statutory analysis
- Policy reasoning
- Compliance
- Law, regulation and compliance
- AI evaluation
- Rubric writing
- Expert review
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Regulatory Compliance AI Evaluator [Remote] in Remote vacancy
$14.5 per hour
A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical...SuggestedHourly payPart timeRemote workFlexible hoursWelo Data
United States4 days ago$14.5 per hour
...AI Web Search Evaluator Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant... ...Indemnity Insurance ~401(k) Retirement Plan Federal Law Compliance In compliance with federal law, all persons hired will...SuggestedHourly payPart timeCurrently hiringImmediate startRemote workWork from home10 hours per weekFlexible hoursWelo Data
United States2 days ago$20 per hour
A tech company specializing in AI is hiring a Digital Web Designer. In this remote role, you will evaluate AI-generated designs and provide feedback to enhance the model’s understanding of aesthetics. An ideal candidate will have a strong background in UI/UX design and...SuggestedRemote workFlexible hoursDataAnnotation
United States4 days ago- ...About the role We are hiring expert Evaluators in Data analysis / quantitative readouts to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise...SuggestedHourly payWork at officeRemote work
Mercor Inc
New York, NY19 hours ago - ...AI Policy Compliance AI Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety team...SuggestedRemote jobHourly payFor contractors10 hours per week
- ...Healthcare Compliance AI Evaluator is a remote clinical-review track for evaluating AI outputs that touch clinical review. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning...Remote jobHourly payFor contractors10 hours per week
- ...Private Equity and M&A AI Evaluator is a remote review track for evaluating AI outputs across... ...reasoning, and policy adherence; flag compliance and reconciliation issues; and document... ...analysis Audit-grade review Regulatory standards Risk assessment Finance...Remote jobHourly payFor contractorsWork experience placement10 hours per week
$20 per hour
A leading AI development company in the United States is seeking detail-oriented individuals for remote opportunities in training AI... ...include developing prompts, writing high-quality responses, and evaluating AI outputs. The ideal candidates are fluent in English with...Hourly payRemote workFlexible hoursSupportFinity
United States4 days ago$20 - $80 per hour
...Role Overview Help improve next-generation AI systems by supplying precise, real-world evaluation, annotation, and feedback. This remote contractor role focuses on how AI models learn, reason, and perform across diverse subject areas. Key Responsibilities Evaluate...Hourly payContract workFor contractorsRemote workSaidGig
United Statesa month ago- ...Summary This is a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible hour... ..., and other content to support AI training datasets. LLM evaluation: reviewing AI-generated responses for accuracy, reasoning quality...Hourly payFor contractorsRemote workFlexible hours
CNTXT AI
Brooklyn, NY13 days ago $30 per hour
...About Prolific Prolific is not just another player in the AI space – we are building the biggest pool of quality human data in... ...and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image generation and...Remote workWork from homeFlexible hoursProlific
United States4 days ago- ...from threats. Job Description The Expert-Level Network Evaluator / System Vulnerability Analyst provides advanced technical research... ...our people are at the center of everything we do. Growth: AI-powered career tool that identifies career steps and learning opportunities...
General Dynamics Information Technology
Maryland, MD8 days ago - ...Digital Health Specialist - Physical Therapist (Remote) - Wheelchair Evaluations Ignite Your Career With CORA! The Digital Health... ...burden. ~ Technology that Works for You - EMR automations and AI-powered tools to save time. What You'll Need:...Full timeRemote work
Cora Physical Therapy
Atlanta, GA13 days ago - ...PhD Mathematics AI Evaluator is a remote review track for evaluating AI outputs across mathematics reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the correct method so the modeling team...Remote jobHourly payFor contractors10 hours per week
- ...Student Feedback AI Evaluator is a remote review track for evaluating AI outputs across student feedback ai specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next...Remote jobHourly payFor contractorsWork experience placement10 hours per week
$30 - $65 per hour
...Help improve next-generation AI systems by evaluating Thai-language audio for fluency, authenticity, and linguistic quality. This remote contractor role uses your Thai language expertise to provide detailed, reliable feedback that strengthens AI understanding and generation...Hourly payFor contractorsRemote workSaidGig
United States1 day ago- ...Indigenous Language AI Evaluator is a remote evaluation track for reviewing indigenous language ai evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling...Remote jobHourly payFor contractors10 hours per week
$43 - $47 per hour
...Role Overview Help improve how advanced AI models respond to sensitive topics in Czech. In this remote, hourly role, you will use Czech language fluency and cultural judgment to evaluate model behavior and support safer, more reliable interactions. Training is provided...Remote jobHourly payImmediate startSaidGig
Europe2 days ago$30 - $65 per hour
...Role Overview Use your Italian language expertise to help improve next-generation AI systems through detailed evaluation of Italian audio. This remote independent contractor role focuses on providing high-quality linguistic input that helps models better understand and...Hourly payFor contractorsRemote workSaidGig
United States1 day ago$80 - $120 per hour
...Privacy / regulatory compliance Evaluator is a remote review track for evaluating AI outputs in privacy law workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can...Remote jobFor contractors10 hours per week$18 - $22 per hour
...Role Overview Help improve the safety of advanced AI models by applying Thai language fluency and cultural judgment to evaluate how models respond to sensitive topics in Thai. Training is provided, and prior AI or machine learning experience is not required. Key Responsibilities...Remote jobHourly payImmediate startSaidGig
Remote2 days ago$12.6 per hour
...Help improve a next-generation AI assistant by completing everyday computer tasks through natural Thai voice interactions, then documenting and evaluating the results. This remote, hourly assignment is designed for native Thai speakers who can work independently on a Mac...Hourly payImmediate startRemote workSaidGig
United States1 day ago$30 - $65 per hour
...Help improve next-generation AI systems by evaluating Dutch-language audio and providing clear, native-level linguistic feedback. This remote contractor role centers on assessing how naturally AI-generated Dutch sounds, with no previous AI experience required. Key Responsibilities...Hourly payFor contractorsRemote workSaidGig
United States19 hours ago$20 per hour
...**Pay Rate:** $20.00 / hour **Location:** Remote (Exclusively for US Citizens) ### About the Role Datatricks AI is actively hiring remote evaluators for **Project Edgehog**, a specialized AI evaluation initiative focused on prompt analysis and visual output quality...Hourly payContract workRemote workFlexible hours$140 per hour
...Role Overview Apply your subject matter expertise to improve how next-generation AI systems learn, reason, and perform. As an AI Domain Expert you will evaluate AI outputs, create challenging prompts, annotate real-world data, and provide concise, actionable feedback...Hourly payPart timeFor contractorsRemote workVisa sponsorshipWork visaFree visaSaidGig
United States11 days ago$14.5 per hour
...remote Search Quality Rating team — and help shape the future of AI-powered search.This is your chance to make a real impact on the... ...every day, all from the comfort of your home.WHAT YOU'LL DO: • Evaluate search results and rate how relevant and useful they are to real...Remote jobBi-weekly payPart time$50 - $75 per hour
...Role Overview Apply your consumer and lifestyle expertise to help evaluate and improve advanced AI systems. Your judgment will help establish the standards used to assess AI responses in areas where practical knowledge, cultural context, and real-world constraints matter...Hourly payFor contractorsLocal areaRemote workSaidGig
United States2 days ago$30 - $65 per hour
...Apply your Vietnamese language expertise to help improve next-generation AI systems through detailed audio evaluations and clear linguistic feedback. This remote contractor role focuses on assessing how naturally Vietnamese is spoken and helping strengthen AI language...Hourly payFor contractorsRemote workSaidGig
United States1 day ago$60 - $65 per hour
...Role Overview Help improve next generation AI language systems by evaluating AI generated Indonesian speech. You will use your native level Indonesian expertise to assess fluency, authenticity, and cultural fit, then provide clear feedback that supports model training...Hourly payFor contractorsRemote workSaidGig
United States10 days ago$30 - $65 per hour
...Help improve next-generation AI systems by evaluating Tamil audio for naturalness, fluency, and linguistic authenticity. This remote contractor role relies on your Tamil language expertise to provide high-quality, real-world feedback that strengthens AI understanding...Hourly payFor contractorsRemote workSaidGig
United States1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Regulatory Compliance AI Evaluator [Remote]. Be the first to apply!
