Everyday AI Agent Tester
$85 per hourTemporary
AuraOne
Category: Enterprise Workflow & Tool Use · Pay: $85 / hr · Location: Remote — US-eligible · Contractor
Everyday AI Agent Tester is a remote review track for evaluating AI outputs across everyday ai agent tester specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it.
Why this role matters
Everyday AI Agent Tester specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done.
Responsibilities
- Review AI outputs against current everyday ai agent tester specialist operations workflows, playbooks, and firm policy for Everyday AI Agent Tester assignments.
- Grade tone, escalation logic, and stakeholder fit on a structured rubric.
- Flag operational risk, missed escalations, and policy-adherence gaps with severity tags.
- Capture the right next step so the modeling team can train on it.
- Adjudicate disputed treatments against published playbooks or firm guidance.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Direct working experience in everyday ai agent tester specialist operations on real teams for Everyday AI Agent Tester work.
- Comfort applying multi-page rubrics consistently across long batches.
- Clear written reasoning that names the policy or workflow being applied.
- Strong attention to detail and the ability to flag when a prompt itself is the problem.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Grade a model's response to a real everyday ai agent tester specialist operations ticket and rate workflow, tone, and escalation.
- Flag a missed escalation with the right severity tag and corrected next step.
- Adjudicate a disputed playbook call between two reviewers using firm guidance.
- Audit a 25-row batch for rubric consistency and report drift to the program lead.
Nice to have
- Prior experience training, calibrating, or QA-ing operations teams.
- Familiarity with AI-assisted workflow tooling and its failure modes.
- Bilingual experience for cross-region operations.
Skills
- Operational review
- Policy adherence
- Workflow judgment
- Stakeholder communication
- Everyday AI Agent Tester specialist operations
- AI model evaluation
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted 14 days ago
Similar jobs that could be interesting for youBased on the Everyday AI Agent Tester in San Jose, CA vacancy
- ...LLC seeks an exceptional Research Scientist Intern to pioneer the next generation of Enterprise Agentic Solutions and Self-Evolving AI Agents. You will work with internal datasets to design, evaluate, and deploy autonomous agents that offload routine workflows in...SuggestedInternship
TikTok USDS Joint Venture
San Jose, CA3 days ago $272k - $431.25k
...seamless, efficient, and secure for millions globally. We seek a Senior Engineer to lead technical efforts in deploying advanced AI agent frameworks and local runtimes on Windows and NVIDIA GeForce RTX GPUs. You will guide development so open-source AI agents (such as...SuggestedFull timeLocal areaShift workNvidia
Santa Clara, CA3 days ago$45 per hour
...frameworks, prompt engineering, model alignment, and intelligent agent systems. Beyond model serving, we operate large-scale log analytics... ...and researchers specializing in Large Language Models and AI Agent systems to join our dynamic team.We are looking for talented...SuggestedHourly payInternshipLocal areaByteDance
San Jose, CA3 days ago- ...DescriptionWho we areMoveworks: the Agentic AI Assistant platform that empowers the... ...including 10% of the Fortune 500, to automate everyday tasks and streamline business operations.... ....Job DescriptionThe RoleMoveworks' AI agents don't just generate text — they act. They...SuggestedContract workWork at officeRemote workFlexible hours
ServiceNow
Santa Clara, CA4 days ago $156k - $387.6k
...Research Scientist Graduate (AI Agent Memory Infrastructure) - 2026 Start (PhD) Location: San Jose Team: Technology Employment Type: Regular About the Team: Join ByteDance’s AI Agent Memory Infrastructure team, where we build the core memory systems that power...SuggestedTemporary workLocal areaByteDance
San Jose, CA2 days ago$218.4k
...server system services.We are looking for an architect-level contributor to lead the design, development, and adoption of enterprise AI agent capabilities, AI harness platforms, RAG-based knowledge base systems, and emerging AI technologies across the organization.This...Temporary workLocal areaOverseasByteDance
San Jose, CA15 hours ago$151.2k - $295.2k
...Team Overview:The team builds the TikTok AI Ad Agent by leveraging AI capabilities to help advertisers run ads with greater ease, confidence, and effectiveness — reducing cognitive, decision, and execution costs across the advertising journey, and eventually driving better...Temporary workLocal areaTikTok
San Jose, CA1 hour ago$20 - $30 per hour
...Immediate ABOUT THE ROLE We evaluate how well advanced AI coding agents solve real engineering problems. An agent is given a real open... ...processes, build failures, and containers. • Workflow — Everyday Git, diffs, pull requests, and issue tracking. • Debugging...Hourly payFull timeContract workImmediate startRemote workBespoke Labs
San Jose, CA5 days ago- ...Product Manager to lead the applied research-to-product pipeline for agent optimization. You will partner with research and engineering to... ...teams. With 12+ years in product management for sophisticated AI or developer products, you will shape roadmaps, engage with...
NVIDIA
Santa Clara, CA1 day ago $218.4k
...Department. We focus on exploring the low-power consumption deployment of AI models, strive for excellence and aim to become an international... ...- Responsible for the technical research and development of AI Agent, improving the application capabilities of large models on XR...Temporary workLocal areaByteDance
San Jose, CA15 hours ago- ...NeMo Helix, NVIDIA’s product for developing, evaluating, deploying, and operating AI systems at scale. This role sits on the Core team responsible for an open source plugin-based agent execution framework and robust APIs. You will design flexible execution systems, mentor...Flexible hours
NVIDIA
Santa Clara, CA1 day ago $128k - $256k
...frameworks, prompt engineering, model alignment, and intelligent agent systems. Beyond model serving, we operate large-scale log analytics... ...and researchers specializing in Large Language Models and AI Agent systems to join our dynamic team. We are looking for talented...Temporary workInternshipLocal areaByteDance
San Jose, CA15 hours ago- ...Job Title: AI Skills and Agents Lead (Low/No code) Start Date: ASAP or Mid-November Duration: 6 months (with possible 6-month extension) Location: Seattle 2-3 days per week or fully remote within the US. Allocation: 100% Full-time Project : AI Workflow...Full timeWork at officeImmediate startRemote work2 days per week3 days per week
Jobrapido
Santa Clara, CA1 day ago - ...Job Overview We are looking for an AI Agent Architect to design and build enterprise internal agents and AI-native workflows. This role requires system architecture, product sense, technical taste, and the ability to navigate ambiguity from problem framing through production...
Maxinsights
Santa Clara, CA4 days ago $388k
...intuition and cutting-edge technology. Come be a part of what’s next.AI and ML power innovation in all areas of the business, including... ...across online inference, batch scoring, feature pipelines, and agent orchestration, so teams can instrument their systems consistently...Hourly payFull timeImmediate startFlexible hoursNetflix
Los Gatos, CA5 days ago$152k - $241.5k
Today, NVIDIA is tapping into the unlimited potential of AI to define the next era of computing, an era in which our GPUs act as the... ...architectures across GPU, LPU, and heterogeneous systems.Developing agents to automate simulation workflows, analyze bottlenecks, and...Full timeNvidia
Santa Clara, CA5 days ago- ...products that accelerate next-generation computing experiences—from AI and data centers, to PCs, gaming and embedded systems. Grounded... ...Technical Program Manager (TPM) to lead execution of the AI Agent roadmap in the Intelligence Systems team within AMD AI Group. This...
AMD
San Jose, CA3 days ago $152k - $241.5k
The Agentic Safety Engineering team is building Agent Policy Fabric (APF), an enterprise governance layer for AI agents. APF combines signed policy, runtime verification, policy projection, credential mediation, and audit records to help govern the actions agents take...Full timeNvidia
Santa Clara, CA4 days ago$180k - $325k
...Software Engineer, Agent Harness & Evaluation – San Jose, CA $180k to $325k We are hiring Software Engineers to improve the performance, reliability, and efficiency of AI agents for hardware applications. You'll identify failure modes, design and run evaluations,...SIVAD Consulting
San Jose, CA4 days ago$212.8k
...Team Introduction We are building the dedicated AI Agent infrastructure and product platform for the Security organization, enabling different business teams to rapidly build, deploy, and scale AI Agents for real-world security use cases. Our mission is to turn the...Temporary workLocal areaByteDance
San Jose, CA15 hours ago- ...VP Engineering, AI Agents - Legal Tech - North America, UK or EU (Eastern Time Zone for US Based Candidates) Vice President, Engineering, Agentic AI We are running a retained search VP Engineering search for a global enterprise technology company building one of the...
Jobrapido
Santa Clara, CA1 day ago $240k - $333k
...Group Product Manager, Finance Agent, AI Workflows Share Group Product Manager, Finance Agent, AI Workflows corporate_fare Google place San Jose, CA, USA ; New York, NY, USA X Note: By applying to this position you will have an opportunity to share your preferred...Google LLC
San Jose, CA2 days ago$176.8k - $286k
...Disruption, Collaboration, Execution, Integrity, and Inclusion. We weave AI into the fabric of everything we do and use it to augment the... ...that. Palo Alto Networks is hiring a Product Manager to build AI agents that handle day-to-day branch operations: finding and fixing...Full timeWork at officePalo Alto Networks, Inc.
Santa Clara, CA1 day ago$51.63k - $124.72k
...Job Description Border Patrol Agent (BPA)\n With more than 69,000 employees, U.S. Customs and Border Protection (CBP) is one of the... ...M-4)\n Skills in Basic Operation of a Motor Vehicle (e.g., everyday use of vehicles, parallel parking)\n \n Other Requirements...Full timeWork experience placementImmediate startNight shiftCustoms and Border Protection
San Jose, CA26 days ago$272k - $431.25k
...enterprise integrations.We are looking for a Principal Software Engineer, Agent Policy Fabric (APF) Core Platform, to join our Cloud Engineering... ...fast in some of the most impactful fields of our generation: AI, Data Engineering, Data Science. If you're a creative engineer...Full timeContract workRemote workNvidia
Santa Clara, CA5 days ago$147k - $237.5k
...Disruption, Collaboration, Execution, Integrity, and Inclusion. We weave AI into the fabric of everything we do and use it to augment the... ...trends; we create them. As a Principal Engineer focused on the Agent, you will be at the center of this mission, tackling "impossible"...Permanent employmentFull timeWork at officeLocal areaPalo Alto Networks
Santa Clara, CA1 day ago$190.9k - $334.1k
...so they could focus on meaningful work. Today, ServiceNow is the AI control tower for business reinvention. Our ServiceNow AI platform... ...people.Job DescriptionThe Forward Deployed Engineer, Voice AI Agent Engineering designs, builds, ships, and operates voice AI agents...Work at officeLocal areaImmediate startRemote workFlexible hoursServiceNow
Santa Clara, CA1 day ago$143.2k - $243.4k
...acquisition closing in March 2026.The combination brings together Veza's AI-native Access Graph with ServiceNow's AI Control Tower and... ...privilege across applications, data, cloud environments, and AI agents. For engineers joining Veza today, this means the scale and resources...Work at officeRemote workFlexible hoursServiceNow
Santa Clara, CA4 days ago$150.4k - $277.6k
...Machine Learning Engineer - LLMs, Agent Systems, and Simulation Tooling, Siri Core Modeling Cupertino, California, United States Machine Learning and AI Join the team redefining what a deeply personal and integrated assistant can be. As part of the Siri organization...Temporary workRelocationApple Inc.
Cupertino, CA1 day ago$156k - $316.8k
...Software Engineer - AI Agent Memory Infrastructure Location: San Jose Team: Technology Employment Type: Regular Job Code: A238905B Responsibilities Design, build, and evolve the next-generation memory infrastructure for AI agents, developing a unified...Temporary workByteDance
San Jose, CA2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Everyday AI Agent Tester. Be the first to apply!
Related searches