Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Agent Tester - Real-World Tasks

Remolity, Inc.

micro1 Operations Featured Jobs Special Referral AI Agent Tester - Real-World Tasks Remote - United States ChatGPT Muse AI agents Attention to Detail Following Instructions Screen Recording Special Referral gives your application an enhanced referral route, helping it stand out and potentially improving your chances of being considered. About the role Role Title: AI Agent Tester - Real-World Tasks AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries. We want to know how well they actually do it, from the point of view of a real user. That’s you. You'll test two AI agents side by side, using your own accounts on your phone, and tell us what worked, what didn’t, and how it felt. Scope of Work Run real-life scenarios: bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms Approve or stop the agent at key steps Screen record every attempt Score each run on completion, quality, control issues, satisfaction, and time Share clear written feedback on the agent experience What you’ll bring US-based, with a Bachelor’s degree Someone who already uses AI in daily life 2-3 years of experience, preferably in engineering, business management, or office/operations Detail-oriented, great at following instructions and documenting each step Generalists welcome, domain experts preferred No coding or AI background needed An active personal AI assistant account (Plus tier), active Muse and Dots access, a smartphone that can screen record, and a payment card you can use for test purchases that are reimbursed afterward Get paid to use the AI tools you already use Work entirely from your phone, on your schedule Have a direct say in how next-generation AI agents get built and improved Schedule and availability Time per task: ~1.5 hours per run (evidence and scoring included). Project kickoff: Already active. Work entirely from your phone, on your schedule. Contract & Payment Terms Onboarding: Within 24-48 hours of being selected, you'll receive the offer and complete onboarding. You'll begin tasking as soon as tasks are assigned. Selection: Apply and complete the screening questions; interview (15-30 minutes max); profile review and presentation to the client; Talent Ops onboarding. Where you can work This role is open to people based in: United States. Remote does not always mean work from any country. Always check the employer’s location and working-hour requirements. #J-18808-Ljbffr Remolity, Inc.

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the AI Agent Tester - Real-World Tasks in New York, NY vacancy
  • micro1 is seeking an AI Agent Tester to evaluate real-world tasks performed by AI agents. This contractor role is US-based and requires testing from your own phone and accounts, with screen recording and detailed feedback on agent performance. You will run scenarios such... 
    Suggested
    For contractors

    micro1

    New York, NY
    2 days ago
  • $25 - $45 per hour

    Role Title: AI Agent Tester — Real-World Tasks Role Type: Contractor Location: U.S based Opportunity Snapshot Pay: $25-45/hour Time per task: ~1.5 hours per run (evidence and scoring included) Project kickoff: Already active. Onboarding: Within 24-48 hours of being selected... 
    Suggested
    For contractors
    Work at office
    Immediate start

    Remotebridge

    New York, NY
    3 days ago
  • Remotebridge is seeking an AI Agent Tester to evaluate real-world tasks from a user perspective. This contractor role operates fully from a smartphone, with timely onboarding and onboarding steps completed within 24-48 hours of selection. You'll run scenarios like bills... 
    Suggested
    For contractors

    Remotebridge

    New York, NY
    3 days ago
  • Role Title: Everyday AI Agent Tester Role Type: Contract Location: U.S based About the role AI agents are starting to handle real-life errands: paying bills, booking trips, doing the groceries...  ...Opportunity Snapshot Time per task: ~1.5 hours per run (evidence and scoring... 
    Suggested
    Remote job
    Contract work
    Work at office
    Weekend work

    Xperteez Technology Pvt Ltd

    New York, NY
    1 day ago
  • Hello Patient is recruiting a technically adept AI Agent Product Manager to own end-to-end delivery of AI agents in real healthcare environments. You will shepherd net-new deployments from scoping through go-live and manage the customer-facing roadmap for each deployment... 
    Suggested

    Urban Ridge Supplies

    New York, NY
    4 days ago
  • $45 per hour

    # Everyday AI Agent Tester$45per hourContractor Remote Last verified 21 Jul 2026Listed on micro1...  ...About the RoleOpportunity Snapshot Time per task: ~1.5 hours per run (evidence and...  ...Monday, October 12## What You'll Do* Run real-life scenarios: bills, returns, email follow... 
    Remote work

    AI Work Expert

    New York, NY
    4 days ago
  • The Zeal Company is seeking an applied AI professional in New York to deploy agent systems in production, solving real customer problems across a portfolio. You will own scoping, architecture, building, and production deployment, working closely with customers to ensure... 

    tz

    New York, NY
    5 days ago
  • $205k - $257k

    Scale has been the leading AI data foundry, helping...  ...reliable AI systems for the world’s most important...  ...Finance vertical within our Agents Data & Reinforcement Learning...  ..., decide what financial tasks are worth modeling, how...  ...AI agents to perform real financial work, and you’... 
    Full time

    Scale AI

    New York, NY
    5 days ago
  • $200k - $300k

     ...Job Description Job Description Real-Time Voice AI Agent Systems Engineer Company: HireNow Staffing (Direct Placement Partner) HireNow...  ...will own the technical foundation behind millions of real-world AI voice interactions—from LiveKit architecture and orchestration... 
    Permanent employment
    Full time
    Relocation
    Visa sponsorship

    HireNow Staffing

    New York, NY
    17 days ago
  • $30 - $90 per hour

    Help improve next-generation AI systems by applying practical expertise...  ...contractor role focuses on creating real-world workflows and feedback that make AI agents more effective in professional...  ...using AI in everyday professional tasks. Use AI agents to research topics... 
    Hourly pay
    For contractors
    Remote work

    SaidGig

    New York, NY
    5 days ago
  •  ...leading security-first enterprise AI company. We build cutting-edge...  ...products that are designed to solve real-world business problems. We’re...  ...-generating LLMs and autonomous agents are revolutionizing how software is built and tasks are automated. At Cohere, we’re... 
    Full time
    Work at office
    Local area
    Remote work
    Home office

    Cohere Inc.

    New York, NY
    5 days ago
  •  ...seeking an Offensive Security Agent Engineer to build the systems...  ...engineering, offensive security, and AI agents, building the harnesses...  ...exploit vulnerabilities in real-world applications. This is an...  ...multi-step offensive security tasks involving reconnaissance, exploitation... 

    PensarAI, Inc.

    New York, NY
    2 days ago
  • $290.4k - $363k

    Scale AI is the data foundation for AI, helping organizations build...  ....About the General Agents TeamThe General Agents team, part...  ...frontier agent development and real-world deployment, translating state-...  ...performance on domain-specific tasks.Interest in shaping the future... 
    Full time

    Scale AI

    New York, NY
    5 days ago
  • $185k - $235k

     ...We combine frontier AI, deep public-sector expertise...  ...ownership - owning core agent-runtime infrastructure...  ...Own outcomes, not just tasks. Identify what needs to...  ...with understanding the real problem. Work across...  .... Build for the real world. Our systems must perform... 
    Full time
    Work at office
    Local area

    Jobot

    New York, NY
    1 day ago
  •  ...Role Title: Everyday AI Agent Tester Role Type: Contractor Location: United States We are seeking detail-oriented U.S.-based professionals to test AI agents performing everyday tasks. Youll evaluate two AI agents side by side document their performance and... 
    Full time
    For contractors
    Work at office
    Remote work

    YO AI Labs

    New York, NY
    10 days ago
  •  ...neurosymbolic foundation model for AI agents that do consequential work,...  ...where the agent has to hold a real company’s rules across...  ...how some corner of the business world operates. \n \n Solutions...  ...role where you own outcomes, not task throughput. \n Not a support... 
    Work at office

    AUI™ (Augmented Intelligence)

    New York, NY
    23 days ago
  • Mirage is an AI video company focused on making creation dramatically...  ...by building multimodal agents that turn creative direction into...  ...tools, and execute creative tasks reliably at scale Integrate state...  ..., evaluation frameworks, and real-world feedback to guide iteration... 
    Full time
    Currently hiring
    Local area
    Night shift

    Mirage

    New York, NY
    4 days ago
  • $180k - $225k

    About Scale AIScale AI is the data foundation for AI, helping organizations...  .... We partner with the world's leading enterprises and...  ...frontier AI systems that solve real business problems.Every day, we...  ...telecommunications to build production AI agents that automate complex workflows... 
    Full time

    Scale AI

    New York, NY
    2 days ago
  • $193.4k - $290k

     ...combining frontier agentic AI, an enterprise-grade...  ...product-market fit and world-class investor support....  ...defining a new category in real time. The work is ambitious...  ...As a Software Engineer, Agents, you'll build the...  ...practical evaluations to drive task completion quality and... 

    Harvey AI

    New York, NY
    6 days ago
  • $150 per hour

     ...durable skills for a changing world of work. We’ve taught tens of thousands...  ...a growing portfolio of AI courses for designers and product...  ...focused on AI workflows and agents for non-technical and semi-technical...  ...be able to look at a messy real-world process and determine where... 
    Contract work
    Part time
    Immediate start

    Designlab

    New York, NY
    1 day ago
  • $99.6k - $174k

    Senior Full Stack Engineer, AI Platform & AgentsBuild...  ..., and deployment of AI agents. We also drive the...  ...agent that transforms the world’s most widely used point...  ...knowledge resource into a real‑time medical assistant....  ...Worked on traditional ML tasks like training, deployment... 
    Full time
    Work at office
    Remote work
    2 days per week

    Wolters Kluwer

    New York, NY
    5 days ago
  • $136k - $187k

    Secure Every Identity, from AI to HumanIdentity is the key to unlocking the potential...  ...relentless drive to solve complex challenges with real-world stakes. We are looking for builders and...  ...'ll join the team designing Okta for AI Agents — one of Okta's newest and most critical... 
    Work experience placement
    Local area
    Worldwide
    Flexible hours
    Shift work

    Okta

    New York, NY
    4 days ago
  • $250k - $300k

     ...Software Engineer (AI Agents) $250,000–$300,000 base + equity New York, NY (Remote) High-growth applied AI company building production...  ...while solving difficult AI problems tied directly to real-world outcomes. About People In AI People In AI connects exceptional... 
    Remote work

    People In AI

    New York, NY
    1 day ago
  •  ...About the Role We're looking for a Senior Software Engineer (AI Agents) to build the core systems behind an intelligent automation...  ...major architectural decisions, and build technology that solves real-world operational problems at scale. What You’ll Do • Build and... 
    Full time

    arcadia

    New York, NY
    2 days ago
  •  ...Experience: 4+ years Focus: AI Product Engineering, Full-Stack...  ...decisions. Their AI agents automate the entire research process...  ...digital twins that simulate real customer behavior, enabling businesses...  ...products used by some of the world's most recognized technology... 
    Full time

    Nxt Level

    New York, NY
    3 days ago
  •  ...We build digital solutions that deliver real impact today and serve as foundations for...  ...possibilities.As a Senior Data Engineer, AI & Agents, you will prepare enterprise domain data...  ...finance, research and development, and real-world data — onto the enterprise data platform.... 
    Contract work

    Appnovation

    New York, NY
    3 days ago
  • $60k - $70k

     ...Job Description AI Agents, Autonomous Decision-Making & Artificial Intelligence - Postdoctoral Researcher \n \n The research...  ...position in AI agents for autonomous decision-making, especially in Real-World Engineering and Scientific Discovery applications. Dr.... 
    Full time
    Immediate start
    Flexible hours

    The City University of New York

    New York, NY
    8 days ago
  • $150k - $230k

     ...Agent Platform Engineer New York City NY On-Site $150000$23000...  ...Software Engineer specializing in AI Agent Infrastructure and...  ...enables AI agents to automate real-world healthcare workflows. Position...  ...workflows and perform complex tasks across multiple systems.... 
    Full time
    H1b
    Visa sponsorship

    Global Placement Firm

    New York, NY
    13 days ago
  • $200k - $265k

    Staff Software Engineer - Agent QualityP-1567At Databricks, we are obsessed...  ...data teams to solve the world’s toughest problems, from security...  ...the world’s best data and AI platform so our customers can focus...  ...quality infrastructure, with real influence over how Databricks measures... 
    Local area
    Worldwide

    DataBricks

    New York, NY
    3 days ago
  • $150k - $190k

    About Traba Traba is building the AI operating layer for the...  ...building applied AI that drives real productivity gains and transforms...  ...join as a founding member of the Agents team and help us build the next...  ...We are on a path to change the world for the better. We create and communicate... 
    Temporary work
    Local area
    Flexible hours
    Shift work

    Traba, Inc.

    New York, NY
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Agent Tester - Real-World Tasks. Be the first to apply!