AI Agent Tester - Remote
YO AI Labs
- Remote job
Job Description
Job Description
Job Title: AI Agent Tester – Real-World Tasks
Role Type: Contractor
Location: United States
We are seeking detail-oriented, U.S.-based AI Agent Testers to evaluate how AI assistants handle everyday tasks. You will test two AI agents using your own accounts and smartphone, assess their performance, and provide feedback to improve their real-world usability.
Key Responsibilities- Test AI agents on tasks such as shopping, bookings, bill payments, returns, email follow-ups, travel, and online forms.
- Approve or stop agents at important decision points.
- Screen-record every task attempt.
- Score performance based on completion, quality, user control, satisfaction, and time.
- Provide clear written feedback on the overall user experience.
- Must be based in the United States.
- Bachelor's degree required.
- 2–3 years of professional experience, preferably in engineering, business management, or operations.
- Regular use of AI tools in daily life.
- Strong attention to detail, written communication, and documentation skills.
- An active personal AI assistant account (Plus tier), active Muse and Dots access, a smartphone with screen-recording capability, and a payment card for reimbursable test purchases.
No coding or AI technical background is required. Generalists are welcome, and domain experts are preferred.
Project Details- Kickoff: Project is already active.
- Onboarding: Within 24–48 hours of selection.
- Work Schedule: Flexible, entirely from your phone.
- Task Duration: Approximately 1.5 hours per run, including evidence and scoring.
- Apply and complete the screening questions.
- Attend a 15–30-minute interview.
- Complete the profile review.
- Receive onboarding instructions from the Talent Operations team.
- Work with AI tools and evaluate real-world performance.
- Enjoy flexible, phone-based work.
- Help shape and improve next-generation AI agents.
- ...: Contractor Location: Global, Fully Remote Time Commitment: Approximately 15 hours... ...Learning Researchers to contribute to an AI training project focused on model development... ...experience with Python and coding agents. You will develop, review, and validate challenging...Remote jobFor contractorsFlexible hours
- ...AI Agent Tester - Real-World Tasks is a remote review track for evaluating AI outputs across ai agent tester real world tasks specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document...Remote jobHourly payFor contractorsWork experience placement10 hours per week
$25 - $45 per hour
...Testers put two AI agents through ordinary errands such as paying bills, shopping, travel booking and form filling, all from a personal phone.... ...documentation. Listed pay: $25 to $45 per hour. Where: Remote, US only. Time: About 1.5 hours per run. Contract work for...Remote workHourly payContract workLive in- ...Job Description Job Description Role Title: Everyday AI Agent Tester Role Type: Contractor Location: United States We are seeking detail-oriented, U.S.-based professionals to test AI agents performing everyday tasks. You'll evaluate two AI agents side by side...Remote jobFor contractorsWork at office
$20 - $30 per hour
...Type: Contract, hourly Location: Remote Hours: 20–30 per week Pay: $20–30/hour, based on experience and language coverage... ...Immediate ABOUT THE ROLE We evaluate how well advanced AI coding agents solve real engineering problems. An agent is given a real open...Remote workHourly payFull timeContract workImmediate start- ...AI Agent Evaluation Specialist This project aims to test and evaluate one AI agent in the development stage. By asking questions and... ...content to the end users and make it safer. This role is fully remote, full-time with a duration of several months (depending on the...Remote workFull time
$40 - $120 per hour
...AI Chatbot Tester $40-120/hr Remote Freelance GENERAL About the Role What if your curiosity and critical eye could help shape the future of artificial intelligence? We're looking for AI Chatbot Testers to put cutting-edge AI systems through their paces — having...Remote workHourly payContract workFreelanceWorldwideFlexible hours- ...next steps. Our partner is looking for a Technical Co-Founder, AI Agents based in United States. Join an early-stage AI venture as a... ...foundations of a high-performing engineering team. This fully remote opportunity is ideal for an entrepreneurial engineer eager to build...Remote workFull time
- ...AI Chatbot Tester MCI is one of the fastest-growing tech-enabled business services companies in the USA, with a strong call center footprint... .... Our contact centers are powered by both on-site and remote agents, leveraging advanced technologies to enhance customer...Remote workTemporary workWork at officeLocal area
$62.4k - $83.2k
...customers across over 100 countries and regions. Job Title: AI Intern, Enterprise Agent Development Supervisor: Head of AI for Industry,... ...: Piscataway, NJ (Onsite preferably, but open to US Remote for the right candidate) Compensation: Targeting $30...Remote workHourly payTemporary workInternshipFlexible hours$185k - $260k
...AI Product Manager, Agents & Orchestration This is a fully remote opportunity and can be worked from any location in the United States. About Us 1mind is building the next generation of revenue teams. We are creating a new category of enterprise software: Autonomous...Remote workFull time- ...Job Description Job Description Role: AI Agentic Tester Location: Denver, CO (Zip Code-80221)- REMOTE Skills: QA/Test Engineering experience Must have-... ...skills needed : Functional Testing Validate AI agent workflows against business requirements. Test...Remote work
$150k - $250k
...People make the decisions that matter. Agents do everything else. Agents write and review... ...start in seconds Location Fully remote in the US. We aren't able to sponsor visas... ...the application questions yourself, without AI. After you apply, we'll tell you how to use...Remote workFull timeWork at officeImmediate startHome officeVisa sponsorship- ...Engineering Manager - AI Agents (Agent Platform, LLMs, Evals) Shape the future of trust in the age of AI At Oscilar, we're building... ...frameworks), AWS Bedrock, Python + Java, MCP This is a remote senior engineering leadership role based in the US. You will report...Remote workFull time
- ...Senior Director, Data Science and AI Agents Location: San Mateo, CA · Nashville, TN · Sterling, VA Open to remote POSITION OVERVIEW The Senior Director of Data Science leads the technical vision, strategy, and execution for AI capabilities that power...Remote work
- ...Everyone has felt this. No one should have to. At Assort Health, we're building one continuous conversation for every patient. An AI agent that knows who you are, remembers how you like to be reached, and carries that context from your first symptom to your full...Remote workWork at officeFlexible hours
- Senior Engineer - AI Voice Agents Location: Chicago, IL (Partial Remote) | Job Type: C2C About the Role Join a dynamic technology team as a Senior Engineer specializing in AI Voice Agents. You will architect, develop, and deploy advanced voice-enabled AI solutions that...Remote workWork from home
- ...Job Title: Member of Technical Staff Agent Infrastructure & Research Job Location: San... ...Company is building infrastructure to make AI provably trustworthy for software development... ...Every sixth week (reflection week) allows remote work. Dealbreakers ~0-to-1...Remote workFull timeInternshipWork at office
$200k - $275k
...Senior Software Engineer, AI Agents Compensation: $200,000 - $275,000 base + equity Location: Remote, USA Join an early-stage applied AI company building the infrastructure and agent layer required to automate complex enterprise knowledge work. The company...Remote workFull time$170k - $225k
...Pros, excellent service for both sides, and an AI-run company that moves faster than its size should allow. About AI Agents at LawnStarter Our mission: power every AI... ...Medical, dental, and vision Location: Fully remote (US). Evals, transcripts, and failure cases reward...Remote workFull timeLocal areaFlexible hours$200k - $320k
...product-focused engineering team building AI-powered tools for data and marketing workflows... ...different LLMs. Build production data agents that use company data to answer analytics... .... Visa sponsorship is available. Location Fully remote in the United States....Remote workFull timeVisa sponsorship$158k - $198k
Director, Conversational AI Agents Join our team at TELUS Digital , a leading digital product consultancy and the digital division of TELUS... ...Work From Near (Hybrid) in a US office or Work From Anywhere (Remote) with travel to clients and TELUS Digital offices expected....Remote workFull timeWork at office$160k - $180k
...About Credo AI The Credo AI Governance Platform operates as the trust layer for global... ...powered by a knowledge graph that is updated by agents, enriched with global regulatory... ...relevant education or training. Location & Remote Culture While this is a remote role and...Remote workFull timeWork at office$126k - $170k
...Technical Product Manager, Identification & JS Agent based in United States. We are looking... ..., experience, and expertise. Fully remote working environment within eligible... ...\n How Jobgether works: We use an AI-powered matching process to ensure your application...Remote workFull time- ...around the world, with over 20 nationalities from our offices and remotely, who truly enjoys their work. We are looking for the most... ...backend engineers to build the core systems behind BJAK's AI Finance Agent. This role is for engineers who can build reliable APIs, workflows...Remote workFull time
- ...Senior Director of Engineering, AI Agents Lehi, Utah, Open to Remote; Remote, US At Podium, we bring AI Employees to local businesses that turn every conversation into revenue. Trusted by 60,000+ businesses across Auto, Home Services, and Aesthetics, Podium captures...Remote workLocal area
$87.7k - $114.2k
...partner is looking for a Software Engineer, Agents for Good based in the United States.... ...contribute to the development of innovative AI-powered products and intelligent agent frameworks... ..., and other business factors. Fully remote and flexible working environment within...Remote workFull timeFlexible hours$126k - $264k
...workloads, embrace innovation, and unleash AI. Our industry-leading developer data... ...generation, AI-powered applications. Atlas Agent Engine is an agent development platform built... ...candidates in the United States based on our remote working model. Position Expectations...Remote workFull timeLocal areaWorldwideFlexible hours$93.82k - $131.5k
...inclusive, adaptable, and forward-thinking organization, apply now. We are currently seeking a Data Engineer Tester - Healthcare Data & AI Validation - Remote to join our team in Philadelphia, Pennsylvania (US-PA), United States (US). Data Engineer Tester (2...Remote workFull timeTemporary workWork at officeFlexible hours- ...and communities. The Role This is not a traditional curriculum developer role - please do not apply if you have dabbled with AI agents. In this unique role, we are accepting applications for two different personas: The AI Solutions Engineer turned Mentor/Coach -...Remote workFull timeWork at officeWork from home
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Agent Tester - Remote. Be the first to apply!
- tsa agent New York, NY
- import export agent New York, NY
- cruise agent New York, NY
- telemarketer - state farm agent team member New York, NY
- agent New York, NY
- showing agent New York, NY
- state farm agent work from home New York, NY
- right of way agent New York, NY
- freight broker agent New York, NY
- title agent New York, NY






