Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Calibration Technology Evaluation Specialist

AI Trainer Jobs

About the role Calibration Technology Evaluation Specialist is a remote review track for evaluating AI outputs across calibration technology evaluation specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it. Calibration Technology Evaluation specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done. Judge tool-use tasks and enterprise agent behavior. Operational judgment is the skill. Responsibilities Review AI outputs against current calibration technology evaluation specialist operations workflows, playbooks, and firm policy for Calibration Technology Evaluation Specialist assignments. Grade tone, escalation logic, and stakeholder fit on a structured rubric. Flag operational risk, missed escalations, and policy-adherence gaps with severity tags. Capture the right next step so the modeling team can train on it. Role details Track Operations & business review Work model Remote · Independent specialist contractor Compensation Hourly rate confirmed after the interview process. Eligible from US What you should bring Direct working experience in calibration technology evaluation specialist operations on real teams for Calibration Technology Evaluation Specialist work. Comfort applying multi-page rubrics consistently across long batches. Clear written reasoning that names the policy or workflow being applied. Strong attention to detail and the ability to flag when a prompt itself is the problem. Reliable async availability for at least 10 hours per week. Role signals Example tasks Grade a model's response to a real calibration technology evaluation specialist operations ticket and rate workflow, tone, and escalation. Flag a missed escalation with the right severity tag and corrected next step. Adjudicate a disputed playbook call between two reviewers using firm guidance. Audit a 25-row batch for rubric consistency and report drift to the program lead. Useful experience Prior experience training, calibrating, or QA-ing operations teams. Familiarity with AI-assisted workflow tooling and its failure modes. Bilingual experience for cross-region operations. Compensation and schedule Hourly rate confirmed after the interview process. Expected arrangement: contractor , with program-defined task volume and review pacing. Placement depends on current program demand and reviewer confirmation. Skills used in matching Operational review Policy adherence Workflow judgment Stakeholder communication Calibration Technology Evaluation specialist operations Application boundary Creating a specialist profile records your experience and preferences. Starting role intake is a separate action that attaches this role to your candidate record. Specialist intake The intake preserves your chosen role, the visible terms, and source attribution for reviewer context. 01 Confirm profile and eligibility details. 02 Attach this role deliberately. 03 Receive a human review decision or follow-up. Placement timing depends on program demand and reviewer confirmation. #J-18808-Ljbffr

Vacancy posted 9 hours ago
Similar jobs that could be interesting for youBased on the Calibration Technology Evaluation Specialist in New York, NY vacancy
  • $50 per hour

     ...English (US) Audio Evaluation Specialist is a remote evaluation track for reviewing audio annotation prompts and responses against AuraOne'...  ...for rubric updates. Maintain reviewer-quality scores by calibrating against gold-standard examples each week.Track Evaluation &... 
    Suggested
    For contractors
    Remote work
    10 hours per week

    AI Trainer Jobs

    New York, NY
    2 days ago
  • $80k - $120k

     ...specialize in delivering innovative automation technologies that improve energy efficiency,...  ...Perform startup, commissioning, calibration, maintenance, and repair of building automation...  ...maintenance, and system performance evaluations. Support software, firmware, and... 
    Suggested
    Full time

    Evergreen Talent Partners

    New York, NY
    2 days ago
  • AuraOne is seeking an OCR and Document Understanding Model Evaluator for a remote evaluation track. Reviewers assess document annotation...  ...model behavior on text, images, and documents, and maintain calibration against gold standards each week. This is a contractor, remote... 
    Suggested
    Remote job
    For contractors

    AI Trainer Jobs

    New York, NY
    2 days ago
  • AuraOne seeks a Hebrew Transcription Expert to remotely evaluate Hebrew outputs using a versioned rubric. You will compare paired responses...  ...include evaluating model outputs, tagging issues, and calibrating against gold standards weekly. Strong Hebrew linguistics and attention... 
    Suggested
    For contractors
    Remote work

    AI Trainer Jobs

    New York, NY
    1 day ago
  • AuraOne is seeking an experienced remote contractor to evaluate instruction tuning reward model outputs according to a versioned rubric...  ...unsafe content, capturing ambiguous prompts, and providing calibration through inter-rater discipline. This is a remote, contractor role... 
    Suggested
    Remote job
    Hourly pay
    For contractors

    AI Trainer Jobs

    New York, NY
    2 days ago
  • AuraOne is seeking a Meeting Audio Conversation Evaluator for a remote contractor role. You will review audio annotation prompts and responses...  ...include comparing outputs, tagging issues, and calibrating against gold standards weekly. #J-18808-Ljbffr AI Trainer Jobs
    Remote job
    For contractors

    AI Trainer Jobs

    New York, NY
    2 days ago
  •  ...label edge cases, and provide structured feedback to retrain models. Responsibilities include evaluating model outputs against a rubric, identifying safety issues, and calibrating scores weekly. The position offers an hourly rate after interview and US-eligible remote... 
    Hourly pay
    For contractors
    Remote work

    AI Trainer Jobs

    New York, NY
    17 hours ago
  • AuraOne is seeking a Tongan Bilingual Expert for a remote evaluation track to review prompts and responses against our quality rubric....  ...cases for the model. You will also judge benchmark failures and calibrate other evaluators as part of the frontier model evaluation team... 
    Remote job
    For contractors

    AI Trainer Jobs

    New York, NY
    2 days ago
  •  ...Trainer Jobs is seeking a Korean bilingual expert for a remote evaluation track. The role involves reviewing Korean generalist prompts...  ...independent contractor, you will assess outputs, tag issues, and calibrate ratings weekly. The position offers an hourly rate to be... 
    Hourly pay
    For contractors
    Remote work

    AI Trainer Jobs

    New York, NY
    18 hours ago
  • AuraOne seeks a contractor to join a remote evaluation track for reviewing writer evaluation prompts and responses against its quality...  ...include evaluating model outputs, tagging issues, and calibrating reviewer quality weekly. Experience with linguistic or trust... 
    Remote job
    For contractors
    10 hours per week
    Flexible hours

    AI Trainer Jobs

    New York, NY
    1 day ago
  • AuraOne is seeking a Chart Understanding Model Evaluator to remotely review chart understanding prompts and model outputs. This independent...  ...include comparing responses, tagging edge cases, and calibrating scores weekly. Strong writing and reliability for 10+ hours per... 
    Remote job
    For contractors
    10 hours per week

    AI Trainer Jobs

    New York, NY
    2 days ago
  • $80k - $95k

     ...government and nonprofit organizations and individuals. Technology Learning Specialist Under the general direction of the IT Training Manager...  ...LMS. Participate in special projects, as required. Evaluate publicly available training materials for possible use by... 
    Full time
    Temporary work
    Work at office
    Remote work

    Katten Muchin Rosenman LLP

    New York, NY
    21 hours ago
  •  ...efficiencies, enhancing processes, and specialist review functions.Provide input into methodologies...  ...and accounting processes.Assist in evaluating GAAP and/or IFRS rules to determine the...  ...cutting-edge AI and breakthrough technologies to collaborate with clients, driving transformative... 
    Work experience placement
    Work at office
    Worldwide
    Flexible hours

    The Bank of New York Mellon

    New York, NY
    5 days ago
  • $140k - $200k

    A Career with Point72's Technology TeamAs Point72 reimagines the future of investing, our Technology team is constantly evolving our firm...  ...risk processes across the enterpriseMentor team members and evaluate emerging capabilities to continuously mature the privileged access... 
    Work experience placement

    Point72

    New York, NY
    5 days ago
  • $90.58k - $114.59k

    Position InformationNY HELP: YesAgency: Mental Health, Office ofTitle: Mental Hygiene Program Evaluation Specialist 3 (NY HELPS), Kingsboro Psychiatric Center, P27987Occupational Category: Health Care, Human/Social ServicesSalary Grade: 23Bargaining Unit: PS&T - Professional... 
    Permanent employment
    Full time
    Work at office
    Remote work

    State of New York, USA

    New York, NY
    2 days ago
  •  ...Events Specialist Midtown Manhattan, New York, NY (Hybrid, 3 days in-office) Company...  ...software M&A—advising on the intersection of technology-driven innovation and business services...  ...vendor coordination. Research and evaluate new venues, restaurants, and event... 
    Full time
    Work at office
    Weekend work
    Afternoon shift

    Confidential Asset Management Organization

    New York, NY
    3 hours ago
  •  ...Job Title: Mainframe IMS DB Tools Specialist Location: New York, NY Work Arrangement...  ...Duration: 12 Months Domain: Enterprise Technology & Mainframe Infrastructure Pay Rate:...  ...ensure optimal system performance • Evaluate change requests, review new versions of... 
    Contract work
    Remote work

    Pacer Group

    New York, NY
    3 hours ago
  • $160k - $180k

    Technology Governance and Controls SpecialistCorporate Job Level: VP | New York, NY | Full...  ...Technology Governance and Controls Specialist serves as a First Line of Defense (1LoD...  ...control testing, control effectiveness evaluations, issue identification, action plan development... 
    Local area

    IDB Bank

    New York, NY
    4 days ago
  • $140k - $160k

     ...seeking a technically strong Senior Systems Specialist, Business Development Platforms to...  ...Firm’s business development and marketing technology. As a member of the Information...  ...vendors, IT, and cross‑functional teams to evaluate enhancements, participate in product roadmaps... 
    Temporary work
    Work at office

    Cravath Swaine & Moore

    New York, NY
    3 days ago
  • AuraOne is seeking a Latin Bilingual Expert for a remote evaluation track. Reviewers compare paired outputs to a quality rubric, label...  ...evaluating outputs against rubrics, tagging issues, and calibrating with gold-standard examples weekly. This contractor role requires... 
    For contractors
    Remote work
    10 hours per week

    AI Trainer Jobs

    New York, NY
    2 days ago
  • $38.47 per hour

     ...management leadership on major tasks or technology assignments as well as establish goals...  ...writing skills Proficient with test and calibration equipment Support user acceptance...  ...AVIXA General CTS (Certified Technical Specialist) is preferred PHYSICAL REQUIREMENTS... 
    Hourly pay
    For contractors
    Local area

    AVI-SPL

    New York, NY
    2 days ago
  • $109.49k - $145.5k

     ...Job Overview The Trade Desk is a global technology company and the world's leading independent platform for digital advertising, with...  ...analysis techniques. Develop and implement analytical frameworks to evaluate cross‑platform advertising exposure, including frequency... 
    Full time
    Temporary work
    Work at office
    Remote work
    Worldwide

    The Trade Desk

    New York, NY
    1 day ago
  • $125.2k - $187.8k

     ...ProfessionalCompany: CitiAre you a senior change specialist looking to shape how a global financial...  ...across business, operations, and technology to drive meaningful, lasting...  ...initiatives, ensuring proposed solutions are evaluated, agreed, and signed off by the appropriate... 
    Full time
    Work at office
    Remote work

    Citigroup

    New York, NY
    5 days ago
  • $70k - $80k

     ...that has a 45 year track record. We’re a technology company that drives huge carbon...  ...\n Role Summary: \n \n The Meter Specialist (Engineering) is a member of Metergy's...  ...the Field Service Engineering Specialist evaluates new technologies, contributes to design... 
    Summer holiday
    Work at office
    Immediate start
    Remote work
    Flexible hours

    Metergy Solutions

    New York, NY
    6 days ago
  • $110k - $130k

     ...Job Description Senior EHR Specialist-Information Technology \n \n Location: \n 620 Foster Ave, Brooklyn, NY 11230 \n \n Hours: \...  ...issues to vendors or consultants when necessary. \n • Evaluate current workflows and partner with department leaders and... 
    Full time
    Temporary work
    Work at office
    Flexible hours
    Afternoon shift

    Premium Health Center

    New York, NY
    6 days ago
  •  ...feedback to retrain models. This is a remote, contractor-based position with US eligibility considerations. The role emphasizes meticulous evaluation, consistent rubric application, and the ability to name issues clearly with rationale. #J-18808-Ljbffr AI Trainer Jobs
    Remote job
    For contractors

    AI Trainer Jobs

    New York, NY
    2 days ago
  • AI Trainer Jobs seeks a Generalist - English & Telugu evaluator to review prompts and responses from AuraOne's data-labeling rubric. You will compare outputs, label edge cases, and provide structured feedback for model retraining. Applicants should have prior evaluation... 
    Remote job
    For contractors
    10 hours per week
    Flexible hours

    AI Trainer Jobs

    New York, NY
    4 days ago
  • YO AI Labs seeks a Humanities Evaluation Specialist to support an AI training project entirely remotely. You will create challenging humanities questions and high-quality answers, and research using authoritative sources with proper citations. Design prompts that require... 
    Remote job

    YO AI Labs

    New York, NY
    4 days ago
  • $79k

    Placement Specialist Who We Are Pathpoint is the full-service E&S wholesaler that combines smart technology and expert E&S guidance to give brokers and agents unprecedented visibility...  ...& assess risks: In this role you will evaluate submissions for completeness, risk... 
    Remote work
    Home office

    Pathpoint, Inc

    New York, NY
    2 days ago
  • $219.1k - $296.5k

    AWS Specialist Technology Team (STT) is the connective tissue between AWS's deep technical specialists, field teams, and customers—delivering...  ...—working through your product managers to shape roadmaps, evaluate trade-offs, and ensure each product delivers measurable impact... 
    Flexible hours

    AmazonWebServices

    New York, NY
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Calibration Technology Evaluation Specialist. Be the first to apply!