Attorney — AI Legal Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Attorney — AI Legal Evaluator is a remote review track for evaluating AI outputs in legal review workflows. Reviewers grade citation accuracy, statutory reasoning, and policy adherence; flag risk; and document the corrected analysis so the modeling team can train on it.
Why this role matters
Legal review content is judged on precision. AuraOne uses qualified legal and policy reviewers to grade AI outputs the way a senior associate would — with citations, severity tags, and the corrected reasoning alongside the original.
Responsibilities
- Review AI outputs against current legal review statutes, case law, or policy for Attorney — AI Legal Evaluator assignments.
- Verify citations, jurisdictions, and dates against authoritative sources.
- Flag risky reasoning, missed exceptions, and policy-adherence gaps with severity tags.
- Capture the corrected analysis or memo paragraph so the modeling team can train on it.
- Adjudicate disputed treatments against published authority or firm policy.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Active or recent qualification in legal review or an adjacent practice for Attorney — AI Legal Evaluator work.
- Hands-on experience drafting, reviewing, or advising on real matters.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that cites authority and names the rule being applied.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Verify the citations in a model's legal review memo and flag any miscites with severity.
- Grade a model's statutory reasoning on a complex fact pattern and rate the analysis.
- Adjudicate a disputed treatment between two reviewers using current authority.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Bar admission or legal-equivalent credential in a relevant jurisdiction.
- Prior experience reviewing AI-assisted legal-research or compliance tooling.
- Bilingual experience for cross-jurisdiction matters.
Skills
- Legal research
- Citation review
- Statutory analysis
- Policy reasoning
- Legal review
- Law, regulation and compliance
- AI evaluation
- Rubric writing
- Expert review
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 9 days ago
Similar jobs that could be interesting for youBased on the Attorney — AI Legal Evaluator [Remote] in Remote vacancy
- A leading AI consultancy firm is seeking legal consultants for project-based opportunities involving testing and improving AI systems. Candidates should have a degree in law and at least 2 years of legal experience within US jurisdiction. Strong written English is required...LegalPermanent employmentFreelanceRemote work10 hours per week
$140 per hour
...Overview Provide high-level M&A legal analysis and strategic input to help train next-generation AI systems. You will apply your M... ...subject-matter expertise to evaluate and generate M&A legal content... ...Preferred: US-qualified attorney with recent experience at a top...LegalRemote jobHourly payFor contractorsSaidGig
Canada2 days ago$65 - $90 per hour
...Role Overview Contribute practical legal expertise to improve AI systems'' understanding and reasoning about legal topics by reviewing AI-generated... ...for legal users. Key Responsibilities Review and evaluate written content and AI-generated responses for legal...LegalRemote jobHourly payPart time10 hours per weekFlexible hoursSaidGig
Remote2 days ago$150 per hour
...Role Overview Evaluate AI-generated legal reasoning across a range of civil justice matters, review realistic client scenarios, identify gaps... ...feedback to improve AI performance. This role is well suited to attorneys with backgrounds in nonprofit advocacy, civil litigation,...LegalHourly payRemote workSaidGig
United States14 hours ago$90 - $130 per hour
...Role Overview Help train and evaluate advanced legal AI by applying your transaction law expertise to contract drafting, redlining, and negotiation scenarios. This part-time contractor role focuses on improving how AI systems identify risk and interpret contract language...LegalHourly payContract workPart timeFor contractorsRemote workSaidGig
United States14 hours ago$90 - $130 per hour
...Overview Provide senior-level legal review, drafting, and analysis... ...used to train next-generation AI systems. This is a remote... ...negotiation and drafting nuances. Evaluate and clarify legal language to... ...~ Open to qualified attorneys who meet the listed education...LegalRemote jobHourly payContract workFor contractorsSaidGig
Remote14 hours ago$140 per hour
...Role Overview Apply your criminal defense expertise to train and evaluate next-generation AI systems. In this contractor role you will provide real-world legal judgment, draft and critique criminal defense content, and help shape how models learn to reason about criminal...LegalRemote jobHourly payFor contractorsSaidGig
Remote1 day ago- ...information.D.R. Horton, Inc. is currently looking for an IT/Privacy Attorney for their Legal Department. The right candidate will review and negotiate... ...legal guidance in connection with the company’s data, AI and privacy policies and procedures. Provide legal guidance...LegalFull timeRemote work
$140 per hour
...Role Overview Provide litigation and transactional legal expertise to help train next-generation AI systems for a prominent client. This remote contractor... ...scenarios to support the development of AI models. Evaluate data privacy and regulatory compliance questions,...LegalRemote jobHourly payFor contractorsSaidGig
Remote2 days ago$14.5 per hour
...diverse, and scalable to supercharge their AI models. As a Welocalize brand, Welo Data... ...? Join our team as a Web Search Evaluator and help shape the future of search engines... ...in the digital landscape, apply today ! Legal Requirements: In compliance with federal...LegalBi-weekly payHourly payPart timeImmediate startRemote workWork from homeFlexible hoursJobs for Humanity
Jacksonville, FL2 days ago$140 per hour
...Role Overview Provide expert healthcare law guidance to shape the training and evaluation of next generation AI systems. You will apply your domain knowledge to review and annotate legal materials, draft and critique sample legal content, evaluate model outputs for legal...LegalRemote jobHourly payContract workFor contractorsSaidGig
Remote1 day ago- A leading AI research firm is seeking experienced legal professionals to contribute to projects that enhance AI systems. You will leverage your expertise in legal research, drafting, and case analysis to develop real-world scenarios for AI testing. Ideal candidates have...LegalFor contractorsRemote work
Turing
Denver, CO4 days ago - A leading AI research accelerator is seeking a Licensed Attorney to enhance AI systems through legal expertise. Responsibilities include writing structured legal analyses, evaluating AI responses, and collaborating on legal research. The ideal candidate has over 4 years...LegalFreelanceRemote work
Turing
Seattle, WA4 days ago $80 - $105 per hour
...Role Overview Help define how advanced AI understands and evaluates fund-related contracts by applying hands-on funds law experience to contract... ...role supports the development of AI systems that improve legal risk identification and contract interpretation for fund formation...LegalHourly payContract workPart timeFor contractorsRemote workSaidGig
United States14 hours ago$90k - $175k
...Senior Family Law Associate / Associate Attorney Needed for Growing Law Firm Salary: $90,000 - $175,000 per... ...Decision Tools which are available at jobot.com/legal. By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Jobot...LegalWork at officeLocal areaWork from home2 days per week1 day per weekJobot
Point Pleasant Beach, NJ3 days ago$90 - $130 per hour
...Role Overview Help shape how advanced legal AI systems read, evaluate, and negotiate deals by applying practical M&A experience to real-world contract review and redlining workflows. You will perform expert contract analysis and create objective evaluation criteria that...LegalHourly payContract workPart timeFor contractorsRemote workSaidGig
United States14 hours ago$80 - $105 per hour
...Apply your M&A drafting and negotiation expertise to train and evaluate advanced AI systems that perform contract review and risk assessment. In... ...redline M&A documents, simulate negotiations, and translate legal judgment into objective feedback that improves model...LegalHourly payContract workPart timeFor contractorsRemote workSaidGig
United States14 hours ago$50 - $60 per hour
...DataAnnotation is looking for an Appellate Attorney to help train AI models by providing complex legal problems and evaluating AI outputs. This role is ideal for General Counsel or those with similar legal expertise. Contract details include working on your own schedule...LegalHourly payContract workRemote workDataAnnotation
Washington DC3 days ago$50 - $60 per hour
...DataAnnotation seeks an experienced Appellate Attorney to train AI models and evaluate their legal outputs. This role allows for flexible project choices and schedules, enabling work from home at an hourly rate of $50-60. Applicants must have a Juris Doctor (J.D.) and...LegalHourly payFor contractorsRemote workWork from homeFlexible hoursDataAnnotation
Florida, NY4 days ago$50 - $60 per hour
...DataAnnotation is looking for an experienced Appellate Attorney to assist in training AI models by evaluating their logical outputs and solving complex legal problems. Ideal candidates will have a J.D. and extensive legal knowledge across various fields, including corporate...LegalHourly payFor contractorsRemote workWork from homeDataAnnotation
Oklahoma City, OK4 days ago$40 - $45 per hour
...Overview 100% REMOTE Contract to Hire Entry Level Attorney / Associate Attorney Needed for Growing Firm!... ...Decision Tools which are available at jobot.com/legal. By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from...LegalHourly payPermanent employmentContract workLocal areaRemote workWork from homeJobot Consulting
Victorville, CA4 days ago$80 - $120 per hour
Mercor is seeking an IP / trademark / copyright law Evaluator with a strong background in relevant legal experience. This remote role involves evaluating AI-generated artifacts and collaborating with AI research teams to enhance quality. The ideal candidate has over 5...LegalRemote jobHourly payWork at office$50 - $60 per hour
DataAnnotation is seeking an Appellate Attorney to train AI models by evaluating their legal outputs and solving complex legal challenges. This role allows for flexible remote work, enabling you to choose projects based on your own schedule and preferences. A J.D. is mandatory...LegalRemote jobHourly payFlexible hours$30 per hour
...Prolific is not just another player in the AI space – we are building the biggest pool... ...as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple... ...models of the future using professional legal expertise. Once you pass our assessment,...LegalRemote workWork from homeFlexible hoursProlific
New York, NY4 days ago$1,750 - $2,150 per month
...technical talent with leading AI research labs. Headquartered in... ...Collaborate with other IP attorneys to design the environment, shape... ...teams to refine task designs and evaluation criteria for IP-law agent... ...writing skills. Compensation & Legal Task Completion Pay:...LegalHourly payFull timeContract workSummer workRemote workMercor
Chicago, IL14 days ago$28 per hour
...will be supporting the overall mission of the United States Attorney's Office (USAO). If you are looking for a meaningful,... ...; oral and written Competence to review, interpret, and evaluate complex legal and non-legal documents Attention to detail and the ability...LegalHourly payFull timeTemporary workWork at officeLocal areaRemote workFlexible hoursFSA Federal
Boston, MA6 days ago- ...Privacy Law AI Evaluator is a remote review track for evaluating AI outputs in privacy law workflows. Reviewers grade citation accuracy,... ...Privacy law content is judged on precision. AuraOne uses qualified legal and policy reviewers to grade AI outputs the way a senior...LegalRemote jobHourly payFor contractors10 hours per week
$200k - $225k
...Consulting Group: Mintz’s Chambers and Legal 500 recognized E‑Data... ...manage search terms, analytics, AI/TAR workflows, and technology‑... ...relationships, including evaluating capabilities, overseeing data... ...candidate grows into the role. E‑Data Attorneys are not expected to progress...LegalWork at officeRemote workMintz, Levin, Cohn, Ferris, Glovsky and Popeo P.C.
Washington DC1 day ago- ...Healthcare Compliance AI Evaluator is a remote clinical-review track for evaluating AI outputs that touch clinical review. Reviewers grade... ...Structured clinical writing Clinical review Legal reasoning Policy review Risk analysis Healthcare Compliance...LegalRemote jobHourly payFor contractors10 hours per week
- ...Data Protection AI Evaluator is a remote evaluation track for reviewing data protection ai evaluation prompts and responses against AuraOne... ...Inter-rater calibration Data Protection AI evaluation Legal reasoning Policy review Risk analysis Data Protection...LegalRemote jobHourly payFor contractors10 hours per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Attorney — AI Legal Evaluator [Remote]. Be the first to apply!
