Remote AI Agent Tester - Real-World Task Evaluator
RemoteJobsOne
RemoteJobsOne is seeking an Everyday AI Agent Tester for a fully remote contractor role based in the United States. You will run real-life scenarios with two AI agents, screen record each attempt, and score results on completion, quality, control issues, and user satisfaction.
Onboarding starts today and you’ll provide clear feedback on the agent experience. The position requires a US-based candidate with a Bachelor’s degree and 2–3 years of relevant experience.
#J-18808-LjbffrVacancy posted 4 hours ago
Similar jobs that could be interesting for youBased on the Remote AI Agent Tester - Real-World Task Evaluator in Jacksonville, FL vacancy
- OpenTrain AI, Inc. is seeking a remote, part-time contractor to test two AI agents on a smartphone, completing realistic errands and recording outcomes. You will evaluate task completion, quality, control issues, user satisfaction, and time, then provide clear feedback....Remote jobPart timeFor contractorsFlexible hours
- ...RemoteJobsOne in the United States is hiring an AI Agent Evaluator on a fully remote, contractor basis. You’ll test two AI agents, run real-life scenarios, and rate completion, quality, and user experience from a real user perspective. Requirements include a US-based...Remote workFor contractors
RemoteJobsOne
Houston, TX4 hours ago - ...micro1 is a data lab that helps AI labs and enterprises train foundational models and agents. We’re hiring an Everyday AI Agent Tester to run real-life scenarios, record outcomes, and provide actionable feedback from a user perspective. You’ll compare two AI agents...Suggested
Aitrainer
Kentucky4 hours ago $25 - $45 per hour
...spam folder. This is a fully remote position open to candidates... ...Pay: $25$45/hr Role Title: AI Agent Evaluator Role Type: Contractor... ...Pay: $25-45/hour Time per task: 1.5 hours per run (evidence... ...agents are starting to handle real-life errands: paying bills booking...Remote workFull timeFor contractorsWork at officeWeekend workRemoteJobsOne
Houston, TX4 days ago$25 - $45 per hour
...This is a fully remote position , open to candidates based in United... ...: $25–$45/hr Role Title: AI Agent Evaluator Role Type: Contractor... ...Pay: $25-45/hour Time per task: ~1.5 hours per run (evidence... ...are starting to handle real-life errands: paying bills, booking...Remote workFor contractorsWork at officeWeekend workRemoteJobsOne
Houston, TX4 hours ago- Mercor is seeking a Real Estate Appraisal — Visual Document Understanding contractor to support AI model evaluation in the real estate domain. This remote, 15-20 hours/week engagement emphasizes rigorous... .... You will review complex tasks, develop evaluation rubrics, and...Remote jobFor contractors
- ...scientists and engineers to evaluate frontier AI models on real technical work, including... ...will design challenging tasks drawn from your practice,... ...them through frontier AI agents and assess results to professional... ...long-form work in a fully remote setting, with explicit...Remote job
- CNTXT AI is seeking a remote contractor to evaluate AI-generated financial content and develop test cases that probe analytical reasoning. You will help improve... ...from balance sheets and reports; creating prompts and Excel-based evaluation tasks; and #J-18808-Ljbffr CNTXT AIRemote jobFor contractors
- ...Obsidian is hiring expert Evaluators in Real estate, hospitality, and events to review AI-generated work for accuracy, rigor, and domain quality. This remote position requires deep expertise and involves grading outputs like documents and presentations. Applicants must...Remote workWork at office
Mercor
San Francisco, CA4 hours ago $60 per hour
...Chemical Engineers to join our Expert Network to train and evaluate AI models using real-world chemical knowledge. A quick 10- to 15-minute test will... ...candidates join Prolific as participants and are paid for tasks. Researchers seeking your expertise may pay up to $60 per...Remote jobHourly payWork from homeFlexible hours$80 per hour
...seeking a part-time QA contributor for an AI project to validate autonomous agents. Candidates must have strong... ...solving. The role includes reviewing tasks, identifying inconsistencies, and defining... ...for AI agents. This flexible remote opportunity allows you to work on your...Remote jobPart timeFlexible hours- Thespian Labs in Somerville, MA seeks an Agent Product Manager to lead our embodied AI products. You will define the roadmap for agent... ...agents that perceive, act, and speak in real environments. You will translate real-world needs into concrete product requirements, measure...
- Archangel Health AI is seeking Clinical AI Evaluators to review AI-generated clinical outputs... ..., and refine responses to real-world medical queries. You will... .... This is a flexible remote role with no patient care... ...United States and earn per task, with no on-call shifts....Remote workFlexible hoursShift work
- Obsidian is hiring expert Evaluators in real estate, hospitality, and events to review AI-generated work products for accuracy and quality. This is a remote position requiring strong expertise in the relevant domains. Ideal candidates should have over 5 years of professional...Remote jobWork at office
- Mercor is a remote contractor role focusing on evaluating AI-generated artifacts within real estate, hospitality, and events. You will assess quality against rubrics, spot factual, presentation, and aesthetic errors, and deliver structured feedback to help improve AI models...Remote jobFor contractorsWork at office
- ...Everyday AI Agent Tester is a remote review track for evaluating AI outputs across everyday ai agent tester specialist operations... ...rules that decide whether a task actually gets done. Responsibilities... ...tester specialist operations on real teams for Everyday AI Agent Tester...Remote jobHourly payFor contractorsWork experience placement10 hours per week
- ...Software Engineering Reviewer to assess AI-generated code, design realistic... ...workflows. You will review AI outputs, create tasks based on real-world scenarios, compare results, develop... ...quality, correctness, and maintainability. Remote work accepted. #J-18808-Ljbffr Planet...Remote work
$225.2k - $394.1k
...that automated a tedious task for his coworker,... ...Today, ServiceNow is the AI control tower for business... ...frontier Ai agent currently in MVP state.... ...intersection of frontier AI and real-world workflow impact. You'll... ...Work personas (flexible, remote, or required in office)...Remote workWork at officeImmediate startFlexible hoursShift workServiceNow
San Diego, CA5 days ago- ...Recruiters / ESR Healthcare is seeking a meticulous Gmail & Google Calendar AI Assistant Evaluator for a remote contract role based in the United States. You will perform real-world digital tasks, give AI clear instructions, and review results to ensure accuracy under...Remote jobContract work
- ...Simulation-to-Real Task Evaluator is a remote evaluation track for reviewing simulation to real task evaluation prompts and responses against AuraOne... ...team can use to retrain. Why this role matters AI data reviewers help turn simulation to real task evaluation...Remote jobHourly payFor contractors10 hours per week
- ...Role Title: Gmail & Google Calendar AI Assistant Evaluator Role Type: Contractor, Remote Location: United States only micro1 is engaging... ...project enhancing AI assistant quality through real-world task evaluation. In this role, you'll use your everyday...Remote workFor contractors
micro1
Kentucky2 days ago $70 - $110 per hour
...technical talent with leading AI research labs.... ...0/hour Location: Remote Duration: 4–6 weeks... ...Role Responsibilities Evaluate AI-generated... ...feedback. Leverage real-world expertise in directing... ...Compensation & Legal ~ Task-based payment upon first...Remote workHourly payContract workSummer workImmediate startMercor
New York, NY16 days ago- ...finally made it possible for AI to impact clinical care... ...an entire ecosystem of real-world evidence to deliver... ...edge AI tools to scale tasks and augment insights.... ...statistical methods, while also evaluating study limitations.... ....Additionally, for remote roles open to individuals...Remote workFull time
Tempus
New York, NY4 days ago $152k - $240k
...operations, gain real-time visibility,... .... Brex’s AI-native automation and world-class service eliminate... ...and accounting tasks for customers so... ...'re building AI agents to automate and... ...per year of fully remote work!... ...sources.Define evaluation frameworks, success...Remote workWork at officeWork from homeBrex
New York, NY6 days ago$25 - $45 per hour
...Skills ChatGPT Muse AI agents Attention to Detail Following... ...agents. We provide frontier evaluations and reinforcement learning... ...Pay: $25-45/hour Time per task: ~1.5 hours per run (evidence... ...are starting to handle real-life errands: paying bills, booking...For contractorsWork at officeWeekend workAitrainer
Kentucky4 hours ago- ...finally made it possible for AI to impact clinical care... ...an entire ecosystem of real-world evidence to deliver... ...edge AI tools to scale tasks and augment insights.... ...statistical methods, while also evaluating study limitations.... ....Additionally, for remote roles open to individuals...Remote workFull time
Tempus
Boston, MA4 days ago - ...Junior Software Engineer for a fully remote, worldwide contractor role. This... ..., and New Zealand. As part of an AI lab partnership, you will evaluate AI-generated code, create coding... ...improve model performance on real‑world software tasks. You will review architecture, debugging...Remote jobFor contractorsWorldwide
$62.4k - $83.2k
...and regions. Job Title: AI Intern, Enterprise Agent Development Supervisor... ...preferably, but open to US Remote for the right candidate)... ...Build capabilities such as task planning, tool calling, and... ...deploy AI Agent solutions in real-world enterprise scenarios....Remote workHourly payTemporary workInternshipFlexible hours$30 - $90 per hour
...Help improve next-generation AI systems by applying... ...operational analysis. This remote contractor role focuses on creating real-world workflows and feedback that make AI agents more effective in professional... ...AI in everyday professional tasks. Use AI agents to research...Remote workHourly payFor contractorsSaidGig
United Statesa month ago- ...MERIT Beauty is seeking Turkish-speaking annotators for a contract role evaluating AI-generated content. You will review content for coherence, consistency, and alignment with real-world expectations in Turkish. The ideal candidates are native or fluent Turkish speakers...Remote workContract workTemporary workImmediate start
MERIT Beauty
United States2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote AI Agent Tester - Real-World Task Evaluator. Be the first to apply!
Related searches
- remote chat agent Jacksonville, FL
- cruise agent Jacksonville, FL
- operations agent Jacksonville, FL
- freight agent no experience Jacksonville, FL
- agent Jacksonville, FL
- work from home chat agent Jacksonville, FL
- airport agent Jacksonville, FL
- commissioning agent Jacksonville, FL
- tsa agent Jacksonville, FL
- import export agent Jacksonville, FL