Remote AI Agent Evaluator QA Strategist
$60 per hourMindrift
Madison, WI
- Remote job
A leading AI company is seeking QAs for autonomous AI agents to ensure quality assurance in complex task evaluations. This remote project-based role is ideal for individuals who excel in analytical thinking and have experience in evaluating system logic. The opportunity offers flexible hours and competitive pay, with potential rates up to $60/hour. Ideal candidates will have strong attention to detail and familiarity with structured data formats. Join the team and help shape the future of AI! #J-18808-Ljbffr Mindrift
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Remote AI Agent Evaluator QA Strategist in Madison, WI vacancy
$80 per hour
A leading AI consultancy is searching for QA analysts to validate AI agents in a project focused on improving complex tasks. This opportunity is a flexible, remote, part-time role ideal for curious thinkers... ...identify inconsistencies in AI evaluation. Competitive compensation...Remote jobPart timeFlexible hours- AuraOne is seeking an Everyday AI Agent Tester specialist operations contractor for remote US-eligible work. You will review AI outputs against workflows, policy, and playbooks, grade tone and escalation, and document next steps for model training. Ideal candidates have...Remote jobFor contractors
$20 - $40 per hour
...seeking detail-oriented contributors to craft realistic AI task scenarios and evaluate agent performance. You will describe steps, judge outcomes, and... ...to help developers improve automation. The role is remote, part-time, hourly, 10-40 hours per week, paid weekly in...Remote jobHourly payWeekly payPart time10 hours per week- HELP SHAPE THE FUTURE OF AI BY EVALUATING HOW WELL AI ASSISTANTS UNDERSTAND USERS, USE PERSONAL CONTEXT, AND DELIVER HELPFUL CONVERSATIONS. Location: Remote Type: Independent contractor / Part-time Project duration: up to 3 months ABOUT THE ROLE At Anyone AI, we create...Remote workPart timeFor contractors
$20 - $40 per hour
Put AI agents through realistic multi-step jobs and judge whether they actually finish them. Your scenarios, ratings and notes show developers... ...to follow detailed guidelines and work independently Contract Remote, hourly contract (independent contractor) Part-time, flexible:...Remote jobHourly payWeekly payContract workPart timeFor contractorsFlexible hours- AuraOne is seeking an AI Agent Evaluator to remotely review and label ai agent evaluation prompts and responses against AuraOne's quality rubric. You will compare outputs, identify edge cases, and provide structured feedback to support retraining. The role focuses on auditing...Remote job
- OpenTrain AI, Inc. is seeking individuals to remotely test two AI agents on everyday tasks using a smartphone. As a part-time contractor in the United States, you will guide agents, screen record sessions, and provide detailed feedback on completion, control, and user...Remote jobPart timeFor contractors
- Appsierra Group is seeking an AI Session Annotator for a fully remote, independent contractor role. You will review recorded AI sessions, compare responses to visuals, and provide precise written feedback using a detailed rubric. Strong English, attention to detail, and...Remote jobWeekly payFor contractorsFlexible hours
- Mindrift is seeking US-licensed physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation project. You will review cases, score diagnostic... ...contact or malpractice exposure. Contributors work remotely, choose tasks, and earn fixed per-task rates plus...Remote jobFor contractorsFlexible hours
- Inworld AI is seeking a voice evaluation specialist who will listen to paired or grouped speech samples, judge naturalness and accuracy, and help refine voices for real-time applications. This role is remote within the United States and involves structured reporting to...Remote job
$35 - $45 per hour
ManpowerGroup seeks a Bilingual Linguist (Spanish/English) for expert evaluation of AI outputs. This remote, part-time role offers 20 hours per week with pay between $35 and $45 per hour (W2). A Mac computer running macOS 15 Sequoia or later is required. You will evaluate...Remote jobHourly payPart time- AI Trainer Jobs is seeking a Pharmacist for a remote review track evaluating AI outputs across pharmacist operations workflows. You will grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk and document next steps for modeling team training...Remote jobFor contractors10 hours per week
- CNTXT, Inc. is seeking a fully remote hourly contractor to support AI data and language projects. You will work on prompts, voice samples, data annotation, LLM evaluation, and localization QA for American English content. The ideal candidate is a native American English...Remote jobHourly payFor contractors
- CNTXT AI is a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible... ...basis. You will perform data annotation, LLM evaluation, content generation, and localization QA, applying senior-level European Portuguese localization...Remote jobHourly payFor contractorsFlexible hours
$30 per hour
...seeking a part-time independent contractor to evaluate AI shopping assistant interactions. The role... ...commit to 20+ hours per week and work remotely on our internal platform. Payments are... ...of $30 per hour. Prior experience in data QA for AI and e-commerce contexts is preferred...Remote jobHourly payPart timeFor contractors- This is a fully remote, hourly contractor role supporting AI data and language projects on a project-based, flexible... ...support AI training datasets. LLM evaluation: reviewing AI-generated responses for... ...diverse topics. Localization QA: ensuring terminology, tone, cultural...Remote workHourly payFor contractorsFlexible hours
- Role Title: AI Product Evaluator Role Type: Contractor Location: India, Japan or United Kingdom, Remote Our team is engaging expert AI Product Evaluators... ...handling AI model and agent evaluations, often within strict... ...digital product, software QA, or AI research projects is...Remote workFor contractors
- ...Role Title: AI Agent Evaluator Role Type: Contractor Location: United States We are seeking detail-oriented professionals to evaluate AI agents performing everyday tasks. Youll test two AI agents side by side assess their performance and provide feedback to...Remote workFull timeFor contractorsWork at office
- ...Category: Frontier Model Evaluation · Pay: Hourly rate confirmed after the interview process · Location: Remote — US-eligible · Contractor AI Agent Evaluator is a remote evaluation track for reviewing ai agent evaluation prompts and responses against AuraOne's quality...Remote workHourly payContract workFor contractors10 hours per week
AuraOne
San Jose, CAa month ago $25 - $45 per hour
...last step is done. If you do not see it check your spam folder. This is a fully remote position open to candidates based in United States. Pay: $25$45/hr Role Title: AI Agent Evaluator Role Type: Contractor Location: U.S based Opportunity Snapshot Pay...Remote workFull timeFor contractorsWork at officeWeekend work$25 - $45 per hour
...as bills, returns, bookings, groceries and travel through two AI agents side by side, using their own accounts on their phone. They approve... ..., step by step. Listed pay: $25 to $45 per hour. Where: Remote, US-based. Time: About 1.5 hours per run, on your own schedule....Remote workHourly payContract workWork at office$80 per hour
A leading AI consultancy is seeking QA professionals for autonomous AI agents focused on validating complex task structures. This flexible, project-based opportunity... ...analytical skills. The role involves reviewing evaluation tasks, identifying inconsistencies, and...Remote jobFlexible hours$60 per hour
...Missouri is seeking contributors for a part-time QA project focused on autonomous AI agents. This flexible remote opportunity requires strong analytical and... ...structured data formats. Candidates will review evaluation tasks, identify inconsistencies, and help define...Remote jobPart timeFlexible hours- Remotebridge is seeking an AI Product Evaluator for remote, contract work across international teams. You will test AI models and agents against defined scenarios, document findings, and help... ...within strict timeframes. Prior QA or AI research experience is a plus. #J...Remote jobContract work
- AuraOne, Inc. is seeking a remote CRM data reviewer to evaluate CRM agent evaluation outputs against a quality rubric, annotate edge cases, and provide structured feedback for retraining. You will compare paired responses, label issues, and document recurring failure modes...Remote jobHourly payFor contractors
- LILT is seeking detail-oriented language reviewers to support AI training and evaluation projects. You will review prompts and AI outputs, annotate... ...across textual and multimedia materials. As a remote contractor, you’ll work on diverse tasks with flexible scheduling...Remote jobFor contractorsFlexible hours
$55 per hour
A leading AI innovation firm is seeking QAs for autonomous AI agents to improve evaluation frameworks. Candidates should possess excellent analytical thinking and attention... ...tasks and define clear standards. This remote, flexible opportunity offers rates up to $55/...Remote jobPart timeFlexible hours$60 per hour
A leading AI firm in Austin is looking for QA experts to validate and improve AI systems. This remote, freelance role requires strong analytical... ...Candidates will review AI evaluation tasks, identify inconsistencies... ...expected behaviors for agents. Ideal applicants have...Remote jobFreelance$55 per hour
A leading AI innovation firm in Dallas is seeking QAs for autonomous AI agents to ensure the quality of complex systems and scenarios. This flexible, remote project is ideal for those with excellent analytical... ...should be adept at evaluating scenarios and documenting findings...Remote jobFlexible hours- Zekai is seeking an AI Product Evaluator for a 100% remote contractor role across global teams. You will apply expertise to train next-generation AI systems by evaluating models and agents against predefined scenarios. You will document findings, reset usage limits, and...Remote jobFor contractors
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote AI Agent Evaluator QA Strategist. Be the first to apply!
Related searches
- agent Madison, WI
- tsa agent Madison, WI
- work from home chat agent Madison, WI
- import export agent Madison, WI
- commissioning agent Madison, WI
- state farm agent Madison, WI
- executive protection agent Madison, WI
- airport agent Madison, WI
- cruise agent Madison, WI
- telemarketer - state farm agent team member Madison, WI