Regulatory Filing AI Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Regulatory Filing AI Evaluator is a remote review track for evaluating AI outputs in legal review workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can train on it.
Why this role matters
Legal review content is judged on precision. AuraOne uses qualified legal and policy reviewers to grade AI outputs the way a senior associate would — with citations, severity tags, and the corrected reasoning alongside the original.
Responsibilities
- Review AI outputs against current legal review statutes, case law, or policy for Regulatory Filing AI Evaluator assignments.
- Verify citations, jurisdictions, and dates against authoritative sources.
- Flag risky reasoning, missed exceptions, and policy-adherence gaps with severity tags.
- Capture the corrected analysis or memo paragraph so the modeling team can train on it.
- Adjudicate disputed treatments against published authority or firm policy.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Active or recent qualification in legal review or an adjacent practice for Regulatory Filing AI Evaluator work.
- Hands-on experience drafting, reviewing, or advising on real matters.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that cites authority and names the rule being applied.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Verify the citations in a model's legal review memo and flag any miscites with severity.
- Grade a model's statutory reasoning on a complex fact pattern and rate the analysis.
- Adjudicate a disputed treatment between two reviewers using current authority.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Bar admission or legal-equivalent credential in a relevant jurisdiction.
- Prior experience reviewing AI-assisted legal-research or compliance tooling.
- Bilingual experience for cross-jurisdiction matters.
Skills
- Legal research
- Citation review
- Statutory analysis
- Policy reasoning
- Legal review
- Legal reasoning
- Policy review
- Risk analysis
- Regulatory
- Filing
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 19 days ago
Similar jobs that could be interesting for youBased on the Regulatory Filing AI Evaluator [Remote] in Remote vacancy
- Mercor is seeking expert Evaluators in Compliance / regulatory response with financial-services AI to review AI-generated outputs (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. This remote, hourly engagement leverages your subject-matter...RegulatoryRemote jobHourly pay
- Obsidian is hiring expert Evaluators for a remote, hourly role focused on Compliance and regulatory response with financial-services AI. You'll assess AI-generated work products for accuracy and domain quality, leveraging your expertise. The ideal candidate has over 5 years...RegulatoryRemote jobHourly payWork at office
- ...Audit and Controls AI Evaluator is a remote review track for evaluating AI outputs across audit workflows. Reviewers grade calculations... ...reviews. Skills Financial analysis Audit-grade review Regulatory standards Risk assessment Audit Quantitative finance...RegulatoryRemote jobHourly payFor contractorsWork experience placement10 hours per week
- About OpenTrain OpenTrain AI is the hiring and contracting organization for this role... ...work 20+ hours per week About AI Response Evaluation AI training is the human side of building... ...use common email, calendar, photo, and file-storage applications and be willing to connect...SuggestedPart timeFor contractorsRemote work
- ...assessment techniques unique to forensic and sex offender populations• Familiarity with individual differences and with psychological regulatory processes affecting individual and interpersonal functioning.• Familiarity with relevant ethical and professional standards of...RegulatoryWork experience placementWork at officeLocal areaRemote work
$14.5 per hour
...AI Web Search Evaluator Welo Data works with technology companies to provide datasets that are high-quality, ethically sourced, relevant, diverse, and scalable to supercharge their AI models. As a Welocalize brand, Welo Data leverages over 25 years of experience in...Bi-weekly payHourly payPart timeImmediate startRemote workWork from homeFlexible hoursWelo Data
United States1 day ago$14.5 per hour
...AI Web Search Evaluator As a Web Search Evaluator, you will play a key role in improving the quality of search engine results, ensuring users find the most relevant and useful information. Your work will directly impact the development of AI algorithms, making search...Hourly payPart timeCurrently hiringImmediate startRemote workWork from home10 hours per weekFlexible hoursWelo Data
United States1 day ago$20 per hour
A tech company specializing in AI is hiring a Digital Web Designer. In this remote role, you will evaluate AI-generated designs and provide feedback to enhance the model’s understanding of aesthetics. An ideal candidate will have a strong background in UI/UX design and...Remote workFlexible hoursDataAnnotation
United States3 days ago- YO AI Labs is seeking an Oncology and Hematology expert to remotely contribute as a contractor... ...to our healthcare AI project. You will evaluate clinical content, review trial data, and... .... In this role you will analyze regulatory documents, assess endpoints, and ensure accuracy...RegulatoryRemote jobFor contractors
- Hippo Enterprises Inc. in Austin, TX seeks a Corporate Paralegal to support holding company governance, regulatory filings, licensing, and compliance. You will manage contracts, prepare agendas/minutes, and coordinate with the regulatory authorities. This role requires...RegulatoryRemote work
- ...behavior biometrics, machine learning, and AI to stop fraud before it happens. Today,... ...sanctions screening, transaction monitoring, and regulatory reporting capabilities. This... ...regulatory reporting systems, including SAR filing and GoAML submissions Collaborate...RegulatoryRemote jobFull timeCasual workImmediate startWorldwideHome officeFlexible hours
Sardine
United States8 hours ago $90 - $100 per hour
...Role Overview Design and author complex regulatory law evaluation tasks for AI systems used in regulatory compliance and government affairs. Workstreams... ...platforms, compliance management systems, and agency filing systems used in regulated industries. Apply regulatory...RegulatoryHourly payFixed term contractRemote workSaidGig
United Statesa month ago$20 - $30 per hour
A remote-focused technology firm is looking for an individual proficient in evaluating large language model outputs. The role involves assessing AI systems, reviewing workflows, and providing actionable feedback to enhance product quality. Ideal candidates will have strong...Remote jobHourly pay- Obsidian is seeking expert Evaluators for a remote, hourly role focused on reviewing AI-generated work products for quality and accuracy. The successful candidates will leverage their subject matter expertise to assess various documents, spreadsheets, and slide decks....Remote jobHourly payWork at office
- YO AI Labs is seeking an Investment & Finance Expert to work remotely as a contractor. The role focuses on evaluating AI outputs related to valuation, financial modeling, markets, and investments, and crafting expert prompts plus reference answers to improve model reasoning...Remote jobFor contractors
- MERIT Beauty is seeking Turkish-speaking annotators for a contract role evaluating AI-generated content. You will review content for coherence, consistency, and alignment with real-world expectations in Turkish. The ideal candidates are native or fluent Turkish speakers...Remote jobContract workTemporary workImmediate start
- ...About the role We are hiring expert Evaluators in Data analysis / quantitative readouts to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise...Hourly payWork at officeRemote work
Mercor Inc
New York, NY4 days ago $100 per hour
...LEGAL SUBJECT-MATTER EXPERT (AI LEGAL WORKFLOWS) ABOUT PARETO.AI Pareto.AI... ...product Participants will create and evaluate realistic legal workflows across: Document... ...contracts, motions, demand letters, regulatory filings, and other legal documents using proper...RegulatoryWork at officeImmediate startRemote workPareto
United States8 hours ago- ...OverviewWe are seeking multiple Senior AI Business Analysts to join our... ...functions to identify, evaluate, and support high-impact AI... ...related to privacy, security, regulatory compliance, and ethical AI standards... ...review of reports, data files and online screens to ensure technical...RegulatoryTemporary workWork experience placementRelocation package
- Feedinkoo is seeking a Graphic Designer to help train and improve AI models focused on UI/UX design, visuals, and user experiences. You will critique AI outputs, provide feedback, and guide model improvements to better reflect aesthetics and usability for designers. The...Remote work
- Obsidian is seeking expert Evaluators in Humanities / arts / culture to review AI-generated work products for accuracy, rigor, and domain quality. You will apply subject-matter expertise to grade outputs and provide structured feedback. This is a remote, hourly engagement...Remote jobHourly payWork at office
- Rex.zone is seeking a Senior AI data annotator to perform data labeling and evaluation for NLP tasks, RLHF assessments, and prompt QA to improve training data quality and model performance. This is a US-based remote, full-time role aligned with Miami talent demand. You...Remote jobFull time
- Turing is seeking detail-oriented AI Analysts based in the United States for a Google Wallet evaluation project. This role allows you to engage with advanced AI tools while contributing to the future of AI. You will evaluate model responses, review output quality, and provide...Remote jobFull timeContract work
- Obsidian is hiring expert Evaluators in Healthcare operations for a remote, hourly engagement. You will review AI-generated work products for accuracy, rigor, and domain quality, leveraging your extensive subject-matter expertise in the field. The ideal candidate has over...Remote jobHourly payWork at office
- Mercor is seeking experts in Spreadsheet QA and workbook maintenance to review AI-generated documents, spreadsheets, and slide decks for accuracy and quality. This is a remote, hourly engagement. Ideal candidates have 5+ years in Spreadsheet QA, fluent English, and strong...Remote jobHourly payWork at office
$14.5 per hour
A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results. This flexible, remote, part-time role focuses on analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical...Remote jobHourly payPart timeFlexible hours- Obsidian is hiring expert Evaluators in real estate, hospitality, and events to review AI-generated work products for accuracy and quality. This is a remote position requiring strong expertise in the relevant domains. Ideal candidates should have over 5 years of professional...Remote jobWork at office
- A leading AI research accelerator is hiring a position focused on contributing to projects that evaluate and enhance AI systems. You will design community service scenarios, write structured explanations, and evaluate AI accuracy. The ideal candidate will have 4+ years...Remote jobFull timeFor contractors
$30 per hour
...Location: Remote Commitment: 10-40 hours/week Role Responsibilities Evaluate outputs from large language models and autonomous agent systems... ...stakeholders. Requirements Strong experience in LLM evaluation, AI output analysis, QA/testing, UX research, or similar analytical...Remote jobHourly payContract work- Mercor is seeking expert Evaluators in Clinical / biomedical / pharma to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. This is a remote, hourly engagement. You will apply deep subject-matter...Remote jobHourly pay
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Regulatory Filing AI Evaluator [Remote]. Be the first to apply!
