Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Calibration Technology Evaluation Specialist

AuraOne

About the role Calibration Technology Evaluation Specialist is a remote review track for evaluating AI outputs across calibration technology evaluation specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational risk; and document the right next step so the modeling team can train on it. Calibration Technology Evaluation specialist operations AI has to fit into an actual day at work. AuraOne uses experienced operators to grade outputs the way a senior peer would — checking workflow, policy, and the unwritten rules that decide whether a task actually gets done. Judge tool-use tasks and enterprise agent behavior. Operational judgment is the skill. Responsibilities Review AI outputs against current calibration technology evaluation specialist operations workflows, playbooks, and firm policy for Calibration Technology Evaluation Specialist assignments. Grade tone, escalation logic, and stakeholder fit on a structured rubric. Flag operational risk, missed escalations, and policy-adherence gaps with severity tags. Capture the right next step so the modeling team can train on it. Role details Track Operations & business review Work model Remote · Independent specialist contractor Compensation Hourly rate confirmed after the interview process. Eligible from US What you should bring Direct working experience in calibration technology evaluation specialist operations on real teams for Calibration Technology Evaluation Specialist work. Comfort applying multi-page rubrics consistently across long batches. Clear written reasoning that names the policy or workflow being applied. Strong attention to detail and the ability to flag when a prompt itself is the problem. Reliable async availability for at least 10 hours per week. Role signals Example tasks Grade a model's response to a real calibration technology evaluation specialist operations ticket and rate workflow, tone, and escalation. Flag a missed escalation with the right severity tag and corrected next step. Adjudicate a disputed playbook call between two reviewers using firm guidance. Audit a 25-row batch for rubric consistency and report drift to the program lead. Useful experience Prior experience training, calibrating, or QA-ing operations teams. Familiarity with AI-assisted workflow tooling and its failure modes. Bilingual experience for cross-region operations. Compensation and schedule Hourly rate confirmed after the interview process. Expected arrangement: contractor , with program-defined task volume and review pacing. Placement depends on current program demand and reviewer confirmation. Skills used in matching Operational review Policy adherence Workflow judgment Stakeholder communication Calibration Technology Evaluation specialist operations Application boundary Creating a specialist profile records your experience and preferences. Starting role intake is a separate action that attaches this role to your candidate record. Specialist intake The intake preserves your chosen role, the visible terms, and source attribution for reviewer context. 01 Confirm profile and eligibility details. 02 Attach this role deliberately. 03 Receive a human review decision or follow-up. Placement timing depends on program demand and reviewer confirmation. #J-18808-Ljbffr

Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Calibration Technology Evaluation Specialist in New York, NY vacancy
  • AuraOne is seeking a Haitian Creole Bilingual Expert to remotely evaluate prompts and responses against our quality rubric. Reviewers...  ...and regression cases for AuraOne Human Data, reviewing frontier model outputs and calibrating criteria weekly. #J-18808-Ljbffr AuraOne
    Suggested
    Remote job
    For contractors

    AuraOne

    New York, NY
    1 day ago
  • AuraOne is seeking a remote contractor to evaluate 360° video recorder outputs against our quality rubric and to provide structured feedback...  ...on grading model outputs, identifying issues, and maintaining calibration through weekly checks. Strong attention to detail and reliable... 
    Suggested
    Remote job
    For contractors

    Auraone

    New York, NY
    3 days ago
  •  .... is seeking a Spanish Bilingual Expert (Mexico) for a remote evaluation track. The role involves reviewing Spanish generalist prompts...  ...reviewer will compare paired outputs, assign severity tags, and help calibrate ratings across batches while documenting recurring failure... 
    Suggested
    Remote job

    AuraOne, Inc.

    New York, NY
    2 days ago
  • AuraOne is seeking a Thai bilingual expert to evaluate generalist evaluation prompts and responses remotely. You will compare outputs...  ...Responsibilities include rating model outputs, tagging content with policy categories, and calibrating against gold #J-18808-Ljbffr Auraone
    Suggested
    Remote job
    For contractors

    Auraone

    New York, NY
    1 day ago
  • AuraOne, Inc. seeks a Haitian Creole Bilingual Expert to remotely evaluate model outputs against a quality rubric. You will label edge...  ...will compare paired responses, document notable failures, and calibrate your assessments against gold-standard examples weekly,... 
    Suggested
    Remote job
    For contractors

    AuraOne, Inc.

    New York, NY
    2 days ago
  • AuraOne is seeking a Rubric Expert to review rubric evaluation prompts and responses, compare outputs, label edge cases, and provide structured...  ...Review frontier model outputs, judge benchmark failures, and calibrate other evaluators. This remote contractor role requires clear... 
    Remote job
    For contractors

    Auraone

    New York, NY
    3 days ago
  •  ...Jobs is seeking a Music & Lyrics Expert - French for a remote evaluation track. Reviewers compare prompts and outputs, label edge cases...  ...generalist outputs against a rubric, write rationales, and help calibrate other evaluators. Requires strong written reasoning, attention... 
    Remote job
    10 hours per week

    Auraone

    New York, NY
    3 days ago
  • AuraOne, Inc. is seeking a remote Video Instruction Following Evaluator to review model outputs and provide structured feedback. You will...  ...rubric. Responsibilities include documenting failure modes, calibrating scores weekly, and routing ambiguous prompts for rubric... 
    Remote job
    For contractors
    10 hours per week

    AuraOne Human Data

    New York, NY
    2 days ago
  •  ...remote, independent contractor to review japanese generalist evaluation prompts and responses against AuraOne's quality rubric. You will...  ...role requires writing clear rationales, labeling issues, and calibrating scores weekly to maintain high reviewer quality. Flexible... 
    Remote job
    For contractors
    Flexible hours

    AuraOne Human Data

    New York, NY
    5 days ago
  •  ...Trainer Jobs is seeking a Tamil Bilingual Expert for a remote evaluation track. Reviewers assess bilingual prompts and model outputs against...  .... You will compare paired responses, label issues, and calibrate scores weekly, with an emphasis on clear written reasoning and... 
    Remote job
    For contractors
    Flexible hours

    AuraOne

    New York, NY
    4 days ago
  •  ...Trainer Jobs is seeking a Korean bilingual expert for a remote evaluation track. The role involves reviewing Korean generalist prompts...  ...independent contractor, you will assess outputs, tag issues, and calibrate ratings weekly. The position offers an hourly rate to be... 
    Remote job
    Hourly pay
    For contractors

    AuraOne

    New York, NY
    1 day ago
  • Agentic Task Completion Model Evaluation Specialist is a remote evaluation track for reviewing agentic task completion model evaluation evaluation...  ...for rubric updates. Maintain reviewer-quality scores by calibrating against gold-standard examples each week. Document... 
    Hourly pay
    For contractors
    Remote work
    10 hours per week

    AuraOne, Inc.

    New York, NY
    3 days ago
  •  ...Jobs is seeking a Haitian Creole Language Expert for a remote evaluation track. You will review language prompts and responses,...  ...include evaluating model outputs, tagging issues, and maintaining calibration with gold-standard examples. Remote, contractor role with hourly... 
    Remote job
    Hourly pay
    For contractors

    Auraone

    New York, NY
    3 days ago
  •  ...seeking a Conlang Language Expert to remotely review language evaluation prompts and responses against our quality rubric. You will...  ...hours per week, and be US-eligible for remote evaluation tasks. Training and rubric calibration are provided. #J-18808-Ljbffr Auraone
    Remote job
    For contractors
    10 hours per week
    Flexible hours

    Auraone

    New York, NY
    3 days ago
  •  ...detail-oriented contractor to review Australian voice actor evaluation prompts and responses against our quality rubric. You will label...  ...written reasoning, consistency across rubrics, and weekly calibration against gold standards. 10 hours per week minimum, with flexible... 
    Remote job
    For contractors
    10 hours per week
    Flexible hours

    Auraone

    New York, NY
    4 days ago
  • AuraOne is seeking an AI Agent Evaluator to remotely review and label ai agent evaluation prompts and responses against AuraOne's quality...  ...model outputs, labeling issues with rubric clauses, and calibrating scores weekly to maintain high reviewer quality. Applicants must... 
    Remote job

    Auraone

    New York, NY
    4 days ago
  • $75 - $110 per hour

     ...Neurology Evaluation Specialist is a remote clinical-review track for evaluating AI outputs that touch neurology. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning so... 
    Hourly pay
    For contractors
    Remote work
    10 hours per week

    AuraOne

    New York, NY
    2 days ago
  •  ...investing in talent development, trading technologies, progressive risk management models,...  .... Our team currently includes over 600 specialists, most of them traders. We are currently...  ...courses). Analytics and effectiveness evaluation: setting up dashboards and reports, monitoring... 

    SFORS

    New York, NY
    1 day ago
  •  ...Agency is seeking someone to create and execute role-play-based evaluation scenarios that simulate realistic customer service interactions for advanced agentic audio models. You will help develop diverse and representative datasets and assess model performance using standardized... 

    Agency.com

    New York, NY
    4 days ago
  • $38.47 per hour

     ...management leadership on major tasks or technology assignments as well as establish goals...  ...writing skills Proficient with test and calibration equipment Support user acceptance...  ...AVIXA General CTS (Certified Technical Specialist) is preferred PHYSICAL REQUIREMENTS... 
    Hourly pay
    For contractors
    Local area

    AVI-SPL

    New York, NY
    1 day ago
  • $77.43k

     ...can by leveraging our platform—which comprises our expertise, technologies, data, brand, talents, and company culture. Across the company...  ...preferential paths beyond what your application demonstrates, and we evaluate every candidate through the same process. All applications... 
    Permanent employment
    Full time
    Contract work
    Fixed term contract
    Remote work
    Relocation package
    Flexible hours

    Bending Spoons

    New York, NY
    20 hours ago
  • $145.92k - $218.88k

     ...investigations, and maintaining business resilience. The Account Technology Strategist (ATS) team brings that value to life by serving as...  ..., technical enablement, and assistance with customer evaluations.Maintain strong knowledge of evolving financial-services regulations... 
    Local area

    cloud software group

    New York, NY
    3 days ago
  • $80k - $95k

     ...government and nonprofit organizations and individuals. Technology Learning Specialist Under the general direction of the IT Training Manager...  ...LMS. Participate in special projects, as required. Evaluate publicly available training materials for possible use by... 
    Temporary work
    Work at office
    Remote work

    Katten Muchin Rosenman LLP

    New York, NY
    2 days ago
  • $89.25k - $150.25k

     ...opportunities to work with the latest technologies, and a commitment to back the broader engineering...  ...-Vulnerability Remediation Management Specialist to join the Remediation team within the...  ...Disclosure Programs.Risk Assessment: Evaluate and prioritize vulnerabilities based on... 

    American Express

    New York, NY
    12 hours ago
  • $110k - $138k

     ...Why WWT? World Wide Technology (WWT) strives to make a new world happen. WWT's work benefits...  ...you be doing? The Computer Vision Specialist is a member of WWT’s Global Solutions &...  ...synthetic data, and on how models will be evaluated, optimized for edge deployment,... 
    Full time
    Work experience placement
    Remote work
    Flexible hours
    Shift work

    World Wide Technology

    New York, NY
    20 hours ago
  • $76k - $105k

     ...-paced, high-visibility environments where technology and service excellence come together? As a Conference Technology Specialist II, you'll be the front line of white glove...  ...training availability, promotion, compensation, evaluation, disciplinary actions, and termination of... 
    Work at office
    Local area
    Worldwide
    Monday to Friday
    Flexible hours

    Kirkland & Ellis LLP

    New York, NY
    20 hours ago
  • $35 - $65 per hour

     ...portfolio of businesses and a strong focus on technology, innovation, and employee experience....  ...for a Technical Learning & Training Specialist to help design, develop, and deliver...  ...programs remain modern and engaging. Evaluate learning solutions for effectiveness and... 
    Hourly pay
    Work at office
    Local area

    Jobot

    New York, NY
    1 day ago
  • evaluation role focused on reviewing video recorder prompts and responses against quality...  ...reliable weekly availability for remote work. Training and calibration will be provided to maintain consistency across batches. #J-... 
    Remote job

    AuraOne Human Data

    New York, NY
    3 days ago
  •  ...reviewer for AuraOne Human Data. This remote contractor role evaluates model outputs, labels edge cases, and writes structured feedback...  ...paired responses, tag hallucinations and unsafe content, and calibrate against gold-standard examples weekly. Strong attention to detail... 
    Remote job
    For contractors

    Auraone

    New York, NY
    3 days ago
  • $66.7k - $106.7k

     ...possibilities purpose BD is one of the largest global medical technology companies in the world. Advancing the world of health is our...  ..., the logistics, timing, strategy, and execution of customer evaluations. Performs product demonstrations, customer education and in... 
    Work at office
    Remote work
    Work from home
    Home office
    Night shift

    Disabledperson, Inc

    New York, NY
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Calibration Technology Evaluation Specialist. Be the first to apply!