Surgical Robotics Task Evaluator [Remote]
AuraOne Human Data
Remote
- Remote job
Surgical Robotics Task Evaluator is a remote review track for evaluating robotics review outputs against real-world physical-AI constraints. Reviewers grade trajectory choices, perception assumptions, and safety envelopes; flag failure modes; and document the corrected behavior so the modeling team can train on it.
Why this role matters
Robotics review systems run in the physical world, where a confident-looking failure can hurt someone. AuraOne uses experienced robotics and operations reviewers to grade outputs against perception, motion, and safety constraints.
Responsibilities
- Review robotics review outputs against current safety standards and operating envelopes for Surgical Robotics Task Evaluator assignments.
- Grade perception, motion, and decision logic on a structured rubric.
- Flag near-miss and unsafe trajectories with severity tags that match incident taxonomies.
- Capture the corrected behavior or reference trajectory so the modeling team can train on it.
- Adjudicate disputed cases against published standards or firm operating procedures.
- Maintain reviewer-quality scores in inter-rater calibration cycles.
Qualifications
- Hands-on experience with robotic, autonomous, or teleoperated systems. Operating, supervising, or evaluating them for Surgical Robotics Task Evaluator work.
- Comfort reasoning about perception, motion planning, and physical safety envelopes.
- Familiarity with structured incident taxonomies and severity scoring.
- Clear written reasoning that names the failure mode and the standard being applied.
- Reliable async availability for at least 10 hours per week.
Example tasks
- Grade a robotics review model's chosen trajectory and flag any near-miss or unsafe motion.
- Score a perception output against ground-truth labels and rate the failure category.
- Adjudicate a disputed safety call between two reviewers using firm operating procedures.
- Audit a 25-trajectory batch for rubric consistency and report drift to the program lead.
Nice to have
- Background in robotics engineering, autonomy operations, or industrial safety.
- Prior experience reviewing AI-assisted robotics tooling and its failure modes.
- Familiarity with simulation environments and trajectory-replay tooling.
Skills
- Robotics evaluation
- Perception review
- Motion safety
- Incident taxonomy
- Robotics review
- Embodied AI
- Safety review
- Surgical
- Robotics
Work model
Remote — US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR. Applicants must be authorized to work from US.
Compensation
Hourly rate confirmed after the interview process.
Application process
Apply through AuraOne's specialist intake for role-specific routing and review. Final project scope, schedule, and contractor terms are confirmed before placement.
Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the Surgical Robotics Task Evaluator [Remote] in Remote vacancy
- Mindrift is seeking US-licensed physicians (MD/DO) to evaluate clinical EHR vignettes for an AI evaluation project. You will review cases... ...or malpractice exposure. Contributors work remotely, choose tasks, and earn fixed per-task rates plus onboarding rewards. #J-188...SuggestedRemote jobFor contractorsFlexible hours
- Mindrift (Toloka) is seeking US-licensed physicians to evaluate clinical EHR vignettes for a medical AI evaluation project. You will review... ...reasoning quality, and write concise rationales. This is a task-based, independent contractor role with fully self-scheduled participation...SuggestedRemote jobFor contractorsFlexible hours
- Robot Trajectory Task Evaluator is a remote evaluation track for reviewing robot trajectory task evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured feedback the modeling...SuggestedHourly payFor contractorsRemote work10 hours per week
- Prolific is seeking fluent Kannada speakers to act as evaluators for AI training tasks conducted remotely with flexible hours. You will listen to audio, compare text and voice, and rate naturalness and cultural accuracy, helping AI understand Kannada nuances. Tasks are...SuggestedRemote jobFlexible hours
$30 per hour
Zekai is seeking a curious, clear writer to help evaluate AI assistants on realistic everyday tasks. You’ll craft short requests, interact with the AI over several turns, and rate the results against a professional standard. This is a 100% remote contract in the United...SuggestedRemote jobHourly payContract work10 hours per weekFlexible hours$20 - $40 per hour
...notes show developers where an agent clicks wrong, skips a step or reports success it did not earn. What you will do Write realistic task scenarios for agents that browse, use apps or handle email and calendars Watch agent runs and record whether each step and the final...Remote jobHourly payWeekly payContract workPart timeFor contractorsFlexible hours- Mindrift is seeking US-licensed physicians (MD/DO) to evaluate written clinical EHR vignettes for an AI evaluation project. This remote, task-based role has no patient contact and emphasizes independent, conscientious review of clinical cases. You will perform independent...Remote jobFlexible hours
- OpenTrain AI, Inc. is seeking individuals to remotely test two AI agents on everyday tasks using a smartphone. As a part-time contractor in the United States, you will guide agents, screen record sessions, and provide detailed feedback on completion, control, and user satisfaction...Remote jobPart timeFor contractors
- Vettedlance seeks PhD scientists in physics, chemistry, biology, microbiology and neuroscience to craft and evaluate research-level AI tasks. You will write questions with verified answers and assess model reasoning against rubrics, handling both physics and biology tracks...Remote jobHourly payContract workPart timeFlexible hours
$20 - $40 per hour
Vettedlance is seeking detail-oriented contributors to craft realistic AI task scenarios and evaluate agent performance. You will describe steps, judge outcomes, and document failures in plain language to help developers improve automation. The role is remote, part-time...Remote jobHourly payWeekly payPart time10 hours per week- ...Travel Research Task Evaluator is a remote review track for evaluating AI outputs across travel research task research review reasoning, calculations, and research workflows. Reviewers grade derivations and assumptions, reproduce key results, and document the correct...Remote jobHourly payFor contractors10 hours per week
$60 per hour
...directly supports our efforts to standardize how visual content is evaluated across different media formats. Establishing a strong baseline... ...Works You will begin by completing a structured calibration task focused on evaluating pairs of photographs. During this process...Remote jobHourly payContract work$60 per hour
...What You'll Do Evaluate pairs of photographs to determine which best meets specific quality standards Assess various images for strict... ...adherence to provided brand guidelines Complete a remote calibration task to demonstrate your visual assessment skills Participate in a...Contract workFor contractorsRemote workTerac
San Francisco, CAa month ago- ...Robot Policy Compliance Task Evaluator is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack scenarios, document the failure mode, and pair each successful jailbreak with the rubric clause it violated so the safety...Remote jobHourly payFor contractors10 hours per week
$85 per hour
...digital assets. How It Works You will complete a 15-20-minute calibration task hosted remotely on our platform. During the session, you will review several image pairs and evaluate how well each matches a set of provided brand guidelines. You will then submit your...Remote jobHourly payContract work- ...Alignerr is seeking Search Quality Evaluators (AI Training) to assess search results, AI-generated answers, and recommendations. This remote... ...accuracy, provide concise written explanations, and complete tasks at your own pace. No prior AI experience required; strong...Hourly payContract workRemote work10 hours per weekFlexible hours
Alignerr Corp.
United States1 day ago - Mindrift seeks MDs or DOs with active US licenses to evaluate written clinical EHR vignettes for an AI evaluation project. You will review cases, rate diagnostic and treatment reasoning, and provide concise rationales. Three independent readings plus adjudication ensure...Remote jobFor contractorsFlexible hours
- ...speaking freelancers based in the United States to support an AI evaluation project focused on assessing the relevance, usefulness,... ...Freelancers may work according to their own schedules for as long as tasks are available. Task availability is based on the client’s business...Contract workFreelanceImmediate start
- ...Telework Eligible No Major Duties As a Program Evaluator you will be responsible for the following duties (If selected at the GS... ...developmental capacity): Serves as the senior evaluator for the Task Force Support (TFS) National Security/Emergency Preparedness (...Full timeContract workTemporary workFor contractorsRemote work
Defense Logistics Agency
Arlington, VA4 days ago $70 - $120 per hour
...Position: 3D Generalist: Modeling, Rigging & Animation (AI Evaluation)] Type: Full-Time Compensation: $70–$120 per hour]... ...to evaluate AI-generated 3D assets, create realistic production tasks, and develop quality standards that improve generative AI models...Hourly payFull timeRemote workWeekday workSynthires
United States2 days ago$26 per hour
Prolific is seeking fluent Danish speakers to act as evaluators for AI training. You will compare text and AI-generated speech, assess naturalness, and flag cultural mismatches. A typical task pays up to $26/hr and can be completed at home with one hour of uninterrupted...Remote job$20.3 per hour
Welo Data is seeking a freelance Maps Visual Design Relevance Evaluator (English, US). This remote role requires access to Google Maps history... ..., readability, and relevance while adhering to strict task instructions. Ideal candidates have an eye for design, spatial...Remote jobHourly payFreelance10 hours per week$50k
CSAA Insurance Group (CSAA IG) is hiring a Casualty Evaluator for a remote role. You will own injury-exposure claims, handle 1st/3rd party... ...multiple states. The role supports homeowners and small-claims tasks within a collaborative, remote-first environment. The position...Remote job- ...Enterprises train foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve LLM... ...suggestions clearly. Adaptability to shifting project priorities and willingness to engage with diverse tasks. Email #J-18808-Ljbffr Micro1For contractorsRemote workShift work
- ...ContractorLocation: RemoteOur team is engaging Cantonese Language Evaluators to contribute to a language-focused project supporting a valued... ...Cantonese speaker, you will participate in internal evaluation tasks using Our team's proprietary system, directly impacting the...
$35.81 - $42 per hour
...Responsibilities Acentra Health is looking for a Part-time PASRR Evaluator to join our growing team. Job Summary Join Acentra Health as a... ...detail and the ability to effectively prioritize and execute tasks in a timely manner Proficiency in Microsoft Office Suite, including...Contract workPart timeWork at officeLocal areaRemote workWork from home- Handshake is seeking Tamil-speaking evaluators based in Singapore or Malaysia to help personalize AI responses. This is not a traditional... ...-time basis. No prior AI training is required. You’ll complete tasks on Handshake’s annotation platform using real-world AI tool experience...Remote jobPart timeFlexible hours
- Social Content Quality Evaluator is a remote evaluation track for reviewing social content quality evaluation prompts and responses against... ...async availability for at least 10 hours per week. Example tasks Compare two social content quality evaluation model responses...Hourly payFor contractorsRemote work10 hours per week
- Zekai is seeking experienced finance professionals fluent in German to train and evaluate AI agents on realistic professional tasks. You will create and review realistic work within digital workspaces, drafting tasks in German and defining success criteria grounded in...Remote jobHourly payPart time
- Prolific is seeking Materials Science Specialists to join its Expert Network for AI training and evaluation. You’ll complete a short test and, if selected, contribute to paid AI tasks requiring one hour of uninterrupted work. Researchers with a background in materials...Work from home
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Surgical Robotics Task Evaluator [Remote]. Be the first to apply!
