Calibration Technology Evaluation Specialist
AuraOne
About the role Calibration Technology Evaluation Specialist is a remote review track for evaluating AI outputs across calibration technology evaluation specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it. Calibration Technology Evaluation specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done. Judge tool-use tasks and enterprise agent behavior. Operational judgment is the skill. Responsibilities Review AI outputs against current calibration technology evaluation specialist operations workflows, playbooks, and firm policy for Calibration Technology Evaluation Specialist assignments. Grade tone, escalation logic, and stakeholder fit on a structured rubric. Flag operational risk, missed escalations, and policy-adherence gaps with severity tags. Capture the right next step so the modeling team can train on it. Role details Track Operations & business review Work model Remote · Independent specialist contractor Compensation Hourly rate confirmed after the interview process. Eligible from US What you should bring Direct working experience in calibration technology evaluation specialist operations on real teams for Calibration Technology Evaluation Specialist work. Comfort applying multi-page rubrics consistently across long batches. Clear written reasoning that names the policy or workflow being applied. Strong attention to detail and the ability to flag when a prompt itself is the problem. Reliable async availability for at least 10 hours per week. Role signals Example tasks Grade a model's response to a real calibration technology evaluation specialist operations ticket and rate workflow, tone, and escalation. Flag a missed escalation with the right severity tag and corrected next step. Adjudicate a disputed playbook call between two reviewers using firm guidance. Audit a 25-row batch for rubric consistency and report drift to the program lead. Useful experience Prior experience training, calibrating, or QA-ing operations teams. Familiarity with AI-assisted workflow tooling and its failure modes. Bilingual experience for cross-region operations. Compensation and schedule Hourly rate confirmed after the interview process. Expected arrangement: contractor , with program-defined task volume and review pacing. Placement depends on current program demand and reviewer confirmation. Skills used in matching Operational review Policy adherence Workflow judgment Stakeholder communication Calibration Technology Evaluation specialist operations Application boundary Creating a specialist profile records your experience and preferences. Starting role intake is a separate action that attaches this role to your candidate record. Specialist intake The intake preserves your chosen role, the visible terms, and source attribution for reviewer context. 01 Confirm profile and eligibility details. 02 Attach this role deliberately. 03 Receive a human review decision or follow-up. Placement timing depends on program demand and reviewer confirmation. #J-18808-Ljbffr
- ...Trainer Jobs is seeking a Tamil Bilingual Expert for a remote evaluation track. Reviewers assess bilingual prompts and model outputs against... .... You will compare paired responses, label issues, and calibrate scores weekly, with an emphasis on clear written reasoning and...SuggestedRemote jobFor contractorsFlexible hours
- AuraOne is seeking a Thai bilingual expert to evaluate generalist evaluation prompts and responses remotely. You will compare outputs... ...Responsibilities include rating model outputs, tagging content with policy categories, and calibrating against gold #J-18808-Ljbffr AuraoneSuggestedRemote jobFor contractors
- AuraOne, Inc. seeks a Haitian Creole Bilingual Expert to remotely evaluate model outputs against a quality rubric. You will label edge... ...will compare paired responses, document notable failures, and calibrate your assessments against gold-standard examples weekly,...SuggestedRemote jobFor contractors
- AuraOne is seeking a Rubric Expert to review rubric evaluation prompts and responses, compare outputs, label edge cases, and provide structured... ...Review frontier model outputs, judge benchmark failures, and calibrate other evaluators. This remote contractor role requires clear...SuggestedRemote jobFor contractors
- AuraOne is seeking a remote contractor to evaluate 360° video recorder outputs against our quality rubric and to provide structured feedback... ...on grading model outputs, identifying issues, and maintaining calibration through weekly checks. Strong attention to detail and reliable...SuggestedRemote jobFor contractors
- ...remote, independent contractor to review japanese generalist evaluation prompts and responses against AuraOne's quality rubric. You will... ...role requires writing clear rationales, labeling issues, and calibrating scores weekly to maintain high reviewer quality. Flexible...Remote jobFor contractorsFlexible hours
- ...Trainer Jobs is seeking a Korean bilingual expert for a remote evaluation track. The role involves reviewing Korean generalist prompts... ...independent contractor, you will assess outputs, tag issues, and calibrate ratings weekly. The position offers an hourly rate to be...Remote jobHourly payFor contractors
- .... is seeking a Spanish Bilingual Expert (Mexico) for a remote evaluation track. The role involves reviewing Spanish generalist prompts... ...reviewer will compare paired outputs, assign severity tags, and help calibrate ratings across batches while documenting recurring failure...Remote job
- ...Jobs is seeking a Music & Lyrics Expert - French for a remote evaluation track. Reviewers compare prompts and outputs, label edge cases... ...generalist outputs against a rubric, write rationales, and help calibrate other evaluators. Requires strong written reasoning, attention...Remote job10 hours per week
- ...Jobs is seeking a Haitian Creole Language Expert for a remote evaluation track. You will review language prompts and responses,... ...include evaluating model outputs, tagging issues, and maintaining calibration with gold-standard examples. Remote, contractor role with hourly...Remote jobHourly payFor contractors
- ...seeking a Conlang Language Expert to remotely review language evaluation prompts and responses against our quality rubric. You will... ...hours per week, and be US-eligible for remote evaluation tasks. Training and rubric calibration are provided. #J-18808-Ljbffr AuraoneRemote jobFor contractors10 hours per weekFlexible hours
- Agentic Task Completion Model Evaluation Specialist is a remote evaluation track for reviewing agentic task completion model evaluation evaluation... ...for rubric updates. Maintain reviewer-quality scores by calibrating against gold-standard examples each week. Document...Hourly payFor contractorsRemote work10 hours per week
- ...detail-oriented contractor to review Australian voice actor evaluation prompts and responses against our quality rubric. You will label... ...written reasoning, consistency across rubrics, and weekly calibration against gold standards. 10 hours per week minimum, with flexible...Remote jobFor contractors10 hours per weekFlexible hours
- AuraOne is seeking an AI Agent Evaluator to remotely review and label ai agent evaluation prompts and responses against AuraOne's quality... ...model outputs, labeling issues with rubric clauses, and calibrating scores weekly to maintain high reviewer quality. Applicants must...Remote job
$75 - $110 per hour
...Neurology Evaluation Specialist is a remote clinical-review track for evaluating AI outputs that touch neurology. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning so...Hourly payFor contractorsRemote work10 hours per week- ...investing in talent development, trading technologies, progressive risk management models,... .... Our team currently includes over 600 specialists, most of them traders. We are currently... ...courses). Analytics and effectiveness evaluation: setting up dashboards and reports, monitoring...
$145.92k - $218.88k
...investigations, and maintaining business resilience. The Account Technology Strategist (ATS) team brings that value to life by serving as... ..., technical enablement, and assistance with customer evaluations.Maintain strong knowledge of evolving financial-services regulations...Local area- ...Agency is seeking someone to create and execute role-play-based evaluation scenarios that simulate realistic customer service interactions for advanced agentic audio models. You will help develop diverse and representative datasets and assess model performance using standardized...
$38.47 per hour
...management leadership on major tasks or technology assignments as well as establish goals... ...writing skills Proficient with test and calibration equipment Support user acceptance... ...AVIXA General CTS (Certified Technical Specialist) is preferred PHYSICAL REQUIREMENTS...Hourly payFor contractorsLocal area- ...reviewer for AuraOne Human Data. This remote contractor role evaluates model outputs, labels edge cases, and writes structured feedback... ...paired responses, tag hallucinations and unsafe content, and calibrate against gold-standard examples weekly. Strong attention to detail...Remote jobFor contractors
$77.43k
...can by leveraging our platform—which comprises our expertise, technologies, data, brand, talents, and company culture. Across the company... ...preferential paths beyond what your application demonstrates, and we evaluate every candidate through the same process. All applications...Permanent employmentFull timeContract workFixed term contractRemote workRelocation packageFlexible hours$89.25k - $150.25k
...opportunities to work with the latest technologies, and a commitment to back the broader engineering... ...-Vulnerability Remediation Management Specialist to join the Remediation team within the... ...Disclosure Programs.Risk Assessment: Evaluate and prioritize vulnerabilities based on...$80k - $95k
...government and nonprofit organizations and individuals. Technology Learning Specialist Under the general direction of the IT Training Manager... ...LMS. Participate in special projects, as required. Evaluate publicly available training materials for possible use by...Temporary workWork at officeRemote work$110k - $138k
...Why WWT? World Wide Technology (WWT) strives to make a new world happen. WWT's work benefits... ...you be doing? The Computer Vision Specialist is a member of WWT’s Global Solutions &... ...synthetic data, and on how models will be evaluated, optimized for edge deployment,...Full timeWork experience placementRemote workFlexible hoursShift work$35 - $65 per hour
...portfolio of businesses and a strong focus on technology, innovation, and employee experience.... ...for a Technical Learning & Training Specialist to help design, develop, and deliver... ...programs remain modern and engaging. Evaluate learning solutions for effectiveness and...Hourly payWork at officeLocal area$76k - $105k
...-paced, high-visibility environments where technology and service excellence come together? As a Conference Technology Specialist II, you'll be the front line of white glove... ...training availability, promotion, compensation, evaluation, disciplinary actions, and termination of...Work at officeLocal areaWorldwideMonday to FridayFlexible hours$66.7k - $106.7k
...possibilities purpose BD is one of the largest global medical technology companies in the world. Advancing the world of health is our... ..., the logistics, timing, strategy, and execution of customer evaluations. Performs product demonstrations, customer education and in...Work at officeRemote workWork from homeHome officeNight shift- Crossing Hurdles is seeking an Image Evaluation Expert for a remote contract role in the United States. You will upload photographs, compare real images with AI outputs, and assess lighting, perspective, and textures to ensure accuracy. You will provide clear, evidence...Remote jobContract work10 hours per week
- Agency is seeking independent Audio Evaluation Specialists for an AI benchmark project focused on agentic audio models. You will design evaluation tasks, run realistic customer-service role-plays, and provide high-quality data. Requirements include native or bilingual...Remote jobHourly payFreelance
- ...benchmarking advanced agentic audio models used in real-world customer support scenarios. You will design and execute role-play-based evaluations across travel, finance, and telecommunications contexts to simulate authentic customer interactions. Contribute to datasets,...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Calibration Technology Evaluation Specialist. Be the first to apply!
- live chat specialist New York, NY
- patient safety specialist New York, NY
- credit card specialist New York, NY
- senior specialist New York, NY
- zoning specialist New York, NY
- charge entry specialist New York, NY
- health program specialist New York, NY
- youth specialist New York, NY
- microsoft specialist New York, NY
- transportation specialist New York, NY




