US-Based Everyday AI Agent Evaluator
Jobleads-US
YO AI Labs is seeking an Everyday AI Agent Tester based in the United States to evaluate AI agents performing common tasks. You will compare two agents side by side, document results, and provide actionable feedback to improve user experience and reliability.
Responsibilities include testing bookings, shopping, travel, emails, bill payments, and forms; deciding when to approve or stop agents; screen-recording sessions; evaluating task completion, quality, user control, and satisfaction; and
#J-18808-Ljbffr Jobleads-USVacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the US-Based Everyday AI Agent Evaluator in New York, NY vacancy
- RemoteJobsOne is seeking an Everyday AI Agent Tester for a fully remote contractor role based in the United States. You will run real-life scenarios with two AI agents... ...on the agent experience. The position requires a US-based candidate with a Bachelor’s degree and 2-3 years...SuggestedRemote jobFor contractorsStart working today
- ...Description Job Description Role Title: AI Agent Evaluator Role Type: Contractor Location:... ...to evaluate AI agents performing everyday tasks. You'll test two AI agents side by... ...usability. Required Qualifications ~ Based in the United States with a bachelor's...SuggestedRemote jobFor contractorsWork at office
YO AI Labs
New York, NY3 days ago - ...Role Title: Everyday AI Agent Tester Role Type: Contractor Location:... ...seeking detail-oriented, U.S.-based professionals to test AI agents... ...everyday tasks. You'll evaluate two AI agents side by side, document... ...begin on Monday, October 12. #J-18808-Ljbffr Jobleads-USSuggestedRemote jobFor contractorsWork at office
$60 per hour
A leading project-based AI consultancy is seeking legal consultants to evaluate AI systems and improve their reasoning. This non-permanent role requires a law degree and at least 2 years of experience in US law. Candidates must have strong written English skills and a...SuggestedPermanent employmentPart timeFreelanceFlexible hours$200k - $275k
...TRM Labs provides AI-powered intelligence... ...deeply involved in evaluating and integrating cutting... ...in the LLM and agent space — including open... ..., and iterate based on user feedback and... ...posting reflect the US base salary only.... ...in getting better everyday. Inspiring Colleague...SuggestedRemote jobLocal areaWorldwide- AuraOne, Inc. is seeking a remote AI data reviewer to evaluate agent evaluation outputs against a versioned rubric. You will label edge cases, provide... ...async work. This contractor role offers flexible remote work and requires US eligibility. #J-18808-Ljbffr AuraOne, Inc.Remote jobFor contractorsFlexible hours
$130 per hour
Turing is seeking US-based dermatologists to review AI-generated dermatology content for accuracy and safety. You will analyze medical images and case summaries, offering structured feedback to improve AI performance. A strong dermatology background and US licensure are...Hourly payContract workPart time$400 per month
...seeking frontend engineers in New York to support an AI project. You will evaluate AI coding models and complete engineering tasks... ...development and the ability to work with AI coding agents. Compensation is task-based at $400 per accepted task, with effective engagement...$99.6k - $174k
Senior Full Stack Engineer, AI Platform & AgentsBuild... ...scale.-Location:: US/Canada, Hybrid or Remote... ..., and deployment of AI agents. We also drive the development... ..., including safety evaluations, iterative testing (e.g... ...range listed is based on primary location of...Full timeWork at officeRemote work2 days per weekWolters Kluwer
New York, NY4 days ago$160k - $210k
BVAL (Bloomberg's Evaluated Pricing Service) Evaluator - US Agency Structured Products Location New York Business Area Product Ref # 1... ...Annual + Benefits + BonusThe referenced salary range is based on the Company's good faith belief at the time of...Price workTemporary workFor contractorsWork experience placementBloomberg
New York, NY2 days ago- About the Opportunity A leading AI research organization is... ...tasks. This project focuses on evaluating how well AI systems handle personalized... ...Strong candidates will have: US-based only Strong MCP experience... ...personal assistants for everyday decisions, but truly useful...Trial period
$15 per hour
...s most innovative companies improve their AI models by providing human feedback. Are you... ...expertise by… Holding a live, scenario-based conversation in English with AI voice models... ...communication and comfortable with natural, everyday conversational speech Bachelor's degree or...Hourly payContract workFreelanceInternshipRemote workFlexible hours$14.5 per hour
Join to apply for the AI Web Search Evaluator role at Welo Data Welo Data works with technology companies... ...process. Hours Set your own schedule based on the following - Minimum commitment... ...understanding of pop culture in the US Reliable computer system and internet...Hourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hours- ...This position will operate in alignment with US timezone. Responsibilities Review... ...of accounts and account holders, evaluate associated risks, and ensure appropriate restriction... ...of proven experience in high-risk, rule-based, or process-driven environments,...Contract workWork from homeFlexible hoursWeekend work
binance
New York, NY4 days ago $29.1k - $41.6k
...Date: Oct 9, 2026 Location: Any city, GA, US, 99999 Work Mode: Virtual (Exception only)... ...development. Summary As a Healthcare Contact Center Agent - Remote US at Gainwell, you can... ...,100.00-$41,600.00 per year, however, the base pay offered may vary depending on geographic...Full timeWork at officeRemote workFlexible hoursGainwell Technologies
New York, NY4 days ago$190 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows,... ...correctly. Qualifications Must-Have US-based only. Strong MCP experience and plugin...Remote jobHourly payContract workFor contractorsSummer workTrial periodMercor
New York, NY9 days ago$265k - $331k
About Scale AIScale AI is the data foundation for AI... ...to build production AI agents that automate complex workflows... ...customers to design, evaluate, and deploy intelligent... ...eligible roles include base salary, equity, and... ...same role. This allows us to ensure a fair and thorough...Full timeScale AI
New York, NY3 days ago$147.6k - $274k
...healthier future. It’s what drives us to innovate. To continuously... ...makes us Roche.Advances in AI, data, and computational sciences... ...Machine Learning Scientist building agents for applied small-molecule drug... ...orchestrate ML models, physics-based methods, and cheminformatics...Full timeLocal areaWorldwideRelocation packageGenentech
New York, NY2 days ago$150k - $190k
...TrabaTraba is building the AI operating layer for the... ...customers are pulling us beyond staffing into the... ...founding member of the Agents team and help us build the... ...product behavior, evaluation systems, and quality frameworks... ..., and rapidly shipping based on direct feedback loops...Temporary workLocal areaFlexible hoursShift workTraba
New York, NY2 days ago$100 - $120 per unit
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position Baseball Fan - Live MLB Game AI Evaluator (US) Type Contract Compensation $100-$120 per task Location Remote Commitment Event-based Role Responsibilities Follow live MLB postseason...Contract workWork experience placementSummer workRemote workAI Model Evaluation. The role is remote for US-eligible contractors, with responsibilities spanning evaluation, rubric-based grading, and documenting corrected approaches for training the modeling team. ... Remote jobFor contractors- Category: Frontier Model Evaluation · Pay: Hourly rate confirmed after the interview... ...process · Location: Remote — US-eligible · Contractor Gmail & Google Calendar AI Assistant Evaluator is a remote... ...Model output evaluation Rubric-based annotation Severity tagging Inter...Hourly payFor contractorsRemote work10 hours per week
$205k - $257k
Scale has been the leading AI data foundry, helping... ...Finance vertical within our Agents Data & Reinforcement... ...that labs use to train and evaluate agents) and the “data as... ...eligible roles include base salary, equity, and... ...same role. This allows us to ensure a fair and thorough...Full timeScale AI
New York, NY4 days ago$114.1k - $268.18k
...KPMG is currently seeking a Manager, SAP AI & Agent Governance to join our Advisory Technology... ...concepts including least privilege, role-based access, delegated authority, segregation... ...be found towards the bottom of our KPMG US Careers site at Benefits & How We Work.Follow...H1bLocal areaKPMG
New York, NY1 day ago$37.5 per hour
...create a golden data set for AI generated ad copy for various... ...role you will be responsible for evaluating advertising copy generated by... ...LocationThis is a Contract position based out of New York, NY.Pay and... ...and Asia, who partner with us for our scale, full-stack capabilities...Contract workTemporary workRemote workTEKsystems
New York, NY14 hours ago- ...innovative, customer service-based approach to logistics. Expeditors... ...love and build a career with us! Job Description Scope of... ...companies, and others) and customers. Evaluate vendors at defined intervals... ...a guide for the team - mentor agents Ensure team meets company...Full timeRemote workOverseas
- .... You will review manuscripts, provide nuanced editorial feedback, evaluate AI-generated work, and help develop editorial guidelines for high publishing standards. Independent contractor engagement; fully remote; US-based contributors preferred. #J-18808-Ljbffr 24-MAGRemote jobFor contractors
- ...with deep expertise in AI and machine learning. Our... ...and Craft Ventures.The Agent Team @ PaveThis role will... ...retrieval systems, and evaluation frameworksStrong product... ...Kingdom — wherever you're based, you'll find the same... ...people notice when they join us — that we truly enjoy...Flexible hours3 days per week
Trove Information Technologies
New York, NY4 days ago $192k - $240k
...Code is Datadog’s cloud coding agent which fixes the problems that Datadog... ...Growth listed above may vary based on the country of your... ...and security platform for the AI era, providing businesses with... .... If you would like to contact us regarding the accessibility of...Work at officeDatadog
New York, NY2 days ago- ...possibilities.As a Senior Data Engineer, AI & Agents, you will prepare enterprise domain data... ...quality, governed access, and reliable agent-based access to trusted data. The ideal... ...experiences and industries to apply. Come join us at Appnovation, and learn more about how...Contract work
Appnovation
New York, NY2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to US-Based Everyday AI Agent Evaluator. Be the first to apply!
Related searches