Compliance Workflow Agent Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Compliance Workflow Agent Evaluator is a remote review track for evaluating AI outputs in compliance workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can train on it.
Why this role matters
Compliance content is judged on precision. AuraOne uses qualified legal and policy reviewers to grade AI outputs the way a senior associate would — with citations, severity tags, and the corrected reasoning alongside the original.
Responsibilities
- Review AI outputs against current compliance statutes, case law, or policy for Compliance Workflow Agent Evaluator assignments.
- Verify citations, jurisdictions, and dates against authoritative sources.
- Flag risky reasoning, missed exceptions, and policy-adherence gaps with severity tags.
- Capture the corrected analysis or memo paragraph so the modeling team can train on it.
- Adjudicate disputed treatments against published authority or firm policy.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Active or recent qualification in compliance or an adjacent practice for Compliance Workflow Agent Evaluator work.
- Hands-on experience drafting, reviewing, or advising on real matters.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that cites authority and names the rule being applied.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Verify the citations in a model's compliance memo and flag any miscites with severity.
- Grade a model's statutory reasoning on a complex fact pattern and rate the analysis.
- Adjudicate a disputed treatment between two reviewers using current authority.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Bar admission or legal-equivalent credential in a relevant jurisdiction.
- Prior experience reviewing AI-assisted legal-research or compliance tooling.
- Bilingual experience for cross-jurisdiction matters.
Skills
- Legal research
- Citation review
- Statutory analysis
- Policy reasoning
- Compliance
- Workflow evaluation
- Tool-use review
- Operational judgment
- Workflow
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 21 days ago
Similar jobs that could be interesting for youBased on the Compliance Workflow Agent Evaluator [Remote] in Remote vacancy
- ...Workflow Escalation Agent Evaluator is a remote evaluation track for reviewing workflow escalation agent evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the...SuggestedRemote jobHourly payFor contractors10 hours per week
- ...Finance Operations Agent Evaluator is a remote review track for evaluating AI outputs across finance and risk workflows. Reviewers grade calculations, narrative reasoning, and policy adherence; flag compliance and reconciliation issues; and document the correct treatment...SuggestedRemote jobHourly payFor contractorsWork experience placement10 hours per week
- ...designed to securely deploy AI agents and automations within... ...empowers employees to streamline workflows, automate repetitive tasks,... ...ensuring data privacy and compliance. North combines cutting-edge... ...a cornerOwn North's agentic evaluation framework, ensuring evals are...SuggestedFull timeWork at officeLocal areaRemote workHome officeFlexible hours
$80k - $95k
...Meet Title: Right-of-Way Acquisition Agent Salary Range: $80,000-$95,000... ...research, prepare documentation, and ensure compliance with legal and regulatory requirements while... ...technology to improve efficiency and workflow. Maintain organized project documentation...SuggestedFull timeWork at officeLocal areaRemote workFlexible hoursEnvironmental Design Group
Columbus, OH25 days ago- ...access, endpoint security & management, and compliance automation—collapsing the stack and... ...build, and evolve our proprietary Windows agent — the software responsible for secure,... ...Participate in Agile development practices, CI/CD workflows, and rigorous code review processes....SuggestedFull timeWork at office3 days per week
I R U, Llc
Remote16 hours ago $50 - $190 per hour
...Compensation: $50–$190/hour Location: Remote Commitment: 20+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows, including personal health , travel , and activity planning . Design realistic prompts for complex...Hourly payContract workFor contractorsSummer workRemote workTrial periodMercor
New York, NY11 days ago- Coinbase is hiring a Group Product Manager to lead the Compliance Agent Experience product team within the EAA organization. You will own end-to-end strategy, drive AI-assisted investigation workflows, and partner with Compliance, Legal, Security, and Procurement to deliver...Remote work
- AI Agent Development & Evaluation Primary Care, Acute Symptom Triage & Travel Health | Contract, up to 10 hrs/week About the Role We are seeking Nurses & Nurse Practitioners with clinical experience in one or more of the following areas: acute symptom management CHF and...Hourly payContract workPart timeRemote work10 hours per weekFlexible hours
- ...Senior AI Architect – Multi-Agent Systems & Platform Infrastructure... ...management, gifting, estate workflows, private investments, and... ...architecture grounded in privacy, compliance, and long-term legacy... ...Develop and refine test plans, evaluation pipelines, and debug tools for...Full timeWork at officeRemote work
Nivalto
San Francisco, CA21 hours ago - KeyCorp seeks an Agentic AI Lead to identify, design, and deploy AI-powered agents transforming banking workflows. You’ll own the end-to-end lifecycle of agent-driven use cases—from discovery to deployment—and partner with Product, Engineering, Risk, and business leaders...Remote job
$109k - $158k
...Summary Manager, Solution Operations - Agent Suite Services within Mastercard is responsible... ...ecosystems are designed for resilience, compliance, and innovation so the team can focus on... ...Delivery Design and evolve technical workflows and governance frameworks that help...Full timeWork experience placementWorldwideMastercard
Canada3 days ago$85 - $95 per hour
...clinicians who split their time between neurodevelopmental diagnostic evaluations and ongoing psychotherapy services. This role allows... ...visit the U.S. Department of Labor’s Office of Federal Contract Compliance Programs (OFCCP) website at . How do you know if you have a...Contract workFor contractorsFor subcontractorWork at officeRemote workFlexible hours1 day per weekWeekday work- About the role We are hiring expert Evaluators in Compliance / regulatory response with financial-services AI to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject...Hourly payWork at officeRemote work
- ...modern frontend solutions using React and TypeScript Develop AI-driven services and intelligent workflows using Python and C# Design, build and integrate AI agents into enterprise applications Use modern AI development tools such as Claude, Cursor and Copilot in...Home officeFlexible hours
Bluebird
Hungary16 days ago $77 per hour
...Certified Special Agent I -Lateral Position:#AC2026 Hiring Salary Statewide: $77,234.... ...inspections of ABC licensed businesses to ensure compliance with required statutes and regulations.... ..., polygraph and psychological evaluation. Criteria: Selected candidates will have...Work experience placementWork at officeLocal areaMonday to FridayRotating shiftWeekend workCommonwealth of VA Careers
Mechanicsville, VA2 days ago$78.54k
...SPECIAL AGENT Job No: 535251 Work Type: Full-time Location: REMOTE OPTIONS, PHOENIX Categories: Legal/Investigations/Adjustment/Compliance ATTORNEY GENERAL (DEPT OF LAW) SPECIAL AGENT Job Location: 15 S. 15th AVE, PHOENIX, AZ 85...Full timeTemporary workWork experience placementWork at officeLocal areaRemote workNight shiftWeekend work2 days per weekArizona Department of Administration
United States2 days ago- Obsidian is seeking expert Evaluators in Privacy / regulatory compliance to review AI-generated work products for accuracy and quality. The role is remote and requires 5+ years of relevant experience along with proficiency in Microsoft Office and Google Workspace. As an...Remote jobHourly payWork at office
$80 per hour
A technology company is seeking a part-time QA contributor for an AI project to validate autonomous agents. Candidates must have strong analytical thinking, attention to detail, and experience with logical problem-solving. The role includes reviewing tasks, identifying...Remote jobPart timeFlexible hours- ...South Dakota. This position entails managing commissioning teams, coordinating with engineering and construction teams, and ensuring compliance with safety and environmental regulations. A minimum of three years in utility-scale electrical systems and strong problem-...Remote work
SOLV Energy
Sioux Falls, SD2 days ago - ...Healthcare Compliance and Patient-Safety Evaluator is a remote clinical-review track for evaluating AI outputs that touch clinical review. Reviewers... ...reasoning, dosing, and guideline adherence in regulated workflows. Responsibilities Review AI outputs against current...Remote jobHourly payFor contractors10 hours per week
$80 - $120 per hour
...Privacy / regulatory compliance Evaluator is a remote review track for evaluating AI outputs in privacy law workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can...Remote jobFor contractors10 hours per week- ...Department provides superior service in all personnel operations in compliance with employment laws and regulations. Through integrity,... ...not necessarily a requirement. There is great demand for phone agents with data entry skills to serve the 24-hour, 7-day a week Call...Full timePart timeWork at officeWork from homeMonday to FridayFlexible hours
- ABRAHAM J. DANIEL CPA is seeking a knowledgeable Enrolled Agent to join a growing firm. You will prepare and review tax returns, represent... ...planning strategies to minimize liabilities while ensuring compliance with federal and state laws. The role emphasizes strong...Remote jobContract workPart time
- ...Compliance Agent The Compliance Specialist is responsible for overseeing compliance operations for a portfolio of affordable housing properties with multiple regulatory programs. This position works closely with site staff, Operations, and the Regional Compliance Manager...Interim roleLocal areaRemote work
- Obsidian is hiring expert Evaluators for a remote, hourly role focused on Compliance and regulatory response with financial-services AI. You'll assess AI-generated work products for accuracy and domain quality, leveraging your expertise. The ideal candidate has over 5...Remote jobHourly payWork at office
- ...Description As Senior Product Manager, Agent Ecosystem, you will own Vanta’s external... .... Your job is to make Vanta the trusted compliance layer that agents rely on. You will define... ...with AI and agent tools, not just evaluating them. ~Experience taking complex products...Full timeWork at officeImmediate startRemote workFlexible hours
Vanta
Remote2 days ago - ...on fact-finding, suitability-based recommendations, ongoing plan reviews, and exceptional client service—while maintaining full compliance with all applicable regulations. This is an opportunity to work with a supportive team and help clients navigate major financial...Work at officeWork from homeFlexible hours
Michael Johnson - State Farm Agent
Chicago, IL16 days ago - Page Mechanical Group, Inc. is looking for a State Designated Broker in Columbia, Maryland, to champion compliance and empower agents. This role ensures brokerage operations meet state laws and provides expert guidance to agents. The ideal candidate should have a valid...Remote job
$110k - $130k
...110,000-$130,000. Description Join Union Pacific as a Special Agent/Police Officer and play a vital role in protecting people, cargo... ...Plan countermeasures as threat levels change, maintaining compliance with federal, state and local regulations. Security and Vulnerability...Local areaShift workNight shift$80 - $120 per hour
...General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: Privacy / regulatory compliance Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-...Contract workSummer workWork at officeRemote workFlexible hoursMercor
Chicago, IL12 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Compliance Workflow Agent Evaluator [Remote]. Be the first to apply!
