AI Model Evaluator & Operational QA Specialist
Crossing Hurdles
Crossing Hurdles is seeking a detail-oriented evaluator to audit AI models and agents across predefined scenarios in a Canadian context. This role focuses on rigorous testing, precise execution, and thorough documentation of results. You will manage repetitive test cycles, ensure adherence to project guidelines, and contribute to building reliable evaluation datasets for AI training platforms. Strong attention to detail and consistent performance under high-volume tasks are essential. #J-18808-Ljbffr Crossing Hurdles
Vacancy posted 22 hours ago
Similar jobs that could be interesting for youBased on the AI Model Evaluator & Operational QA Specialist in New York, NY vacancy
- Mercor Operations is seeking a board-certified radiologist (MD or DO) to evaluate health AI models. You will read de-identified patient charts and judge AI-provided radiology answers for clinical accuracy. This is a fully remote, part-time, hourly role open to qualified...SuggestedRemote jobHourly payPart timeFor contractors
$15 per hour
...Get paid to spot what AI-generated media gets wrong Hold live conversations with AI voice models, compare image and video pairs, and explain what’s off. Fully remote... ..., learn, and improve. Through Outlier’s evaluation and feedback loop, your work helps turn frontier...SuggestedWeekly payContract workFreelanceInternshipRemote workFlexible hoursOutlier
New York, NY22 hours ago- AI Trainer Jobs seeks a remote independent contractor to review AI outputs for fintech operations evaluation. You will assess workflow adherence, tone, and escalation logic, assigning severity... ...tags and documenting next steps for model training. The role requires experience...SuggestedRemote jobHourly payFor contractorsFlexible hours
- Mercor is seeking board-certified critical care physicians (MDs/DOs) to evaluate health AI models in partnership with a leading AI lab. You will read de-identified patient charts and judge AI answers to clinical questions in critical care. Responsibilities include evaluating...SuggestedRemote jobHourly payPart timeImmediate start
- DataAnnotation is seeking experienced computational biologists and bioinformaticians to help train AI models. You’ll design realistic analysis tasks from your practice, run them through frontier AI systems, and grade the results against professional standards. The role...SuggestedRemote job
- Mercor is seeking experienced MDs or DOs to evaluate health AI models remotely in the United States. You will review outputs across diagnoses, clinical reasoning and care workflows, and provide structured feedback to ensure medical accuracy and safety. The role requires...Remote jobPart timeFor contractors
$35 - $40 per hour
A leading evaluation organization seeks a Generalist Evaluator Expert for a remote, flexible... ...involves designing prompts for language model evaluations, defining standards, and conducting... ...thinking skills, familiarity with AI tools like ChatGPT, and a BS or BA degree...Remote jobContract workFlexible hours- ...seeks a Data Provenance Data Specialist contractor to review AI outputs and ensure... ...escalation rules. You will evaluate tone, eligibility, and stakeholder... ...next steps for the modeling team, and calibrate... ...experience with data provenance operations, and clear written...Remote jobFor contractors
$159.38k - $311.68k
...more sustainable, more inclusive world. About the Job You’re Considering As AI reshapes how organizations operate, compete, and create value, we are evolving our consulting model to be AI-enabled by design, augmenting human judgment with analytics, automation, and...Permanent employmentFull timeLocal area- ...is seeking a remote contractor to review AI outputs for Electrical Hardware Engineering... ...and capture the right next steps for the modeling team. This role requires direct... ...Familiarity with multi-page rubrics and cross-region operations is a plus. #J-18808-Ljbffr AuraOneRemote jobFor contractors10 hours per week
- QA Specialist - AI Product Testing & Support is a remote review track for evaluating AI outputs across qa ai product testing support specialist operations workflows. Reviewers grade workflow correctness, policy... ...right next step so the modeling team can train on it. Why...Hourly payFor contractorsWork experience placementRemote work10 hours per week
$85 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated implementations involving cloud platforms...Contract workSummer workRemote work- Create challenging prompts and evaluate large language model responses for accuracy, reasoning, and usefulness. This worldwide,... ...performance management, workforce planning, and operational execution to assess how well AI handles management-related topics. Create complex...Contract workPart timeFor contractorsRemote workWorldwide
$30 - $49 per hour
AI Generalist Digital Workflow Evaluator Review AI-generated outputs and workflow screenshots, apply project guidelines... ...help evaluate AI workflows and model outputs. Your reviews will use... ...experience in business, technology, operations, or another field. Hands-on experience...Hourly payContract workPart timeFor contractorsRemote workFlexible hours- Biz Tech Consultants seeks a surgical AI evaluator to verify procedural, anatomical and clinical accuracy of AI-generated content used in operative settings. The role reviews pre- and post-operative protocols, decision-making, and flags patient-safety risks, ensuring outputs...
- ...remote, hourly contractor role, you will evaluate AI-generated financial content and develop... ...work directly improves how leading AI models handle financial information, making... ...data across balance sheets, reports, and operational records Developing prompts and test cases...Remote jobHourly payFull timeFor contractors
- AI Trainer Jobs is seeking a remote contractor to review AI-generated outputs for Revenue Accounts Receivable Manager operations. You will assess alignment with workflows, policies, and escalation... ...the next steps to train the modeling team. Strong attention to detail...Remote jobFor contractors10 hours per week
- ...contractor to conduct Customer Experience Operations reviews for India assignments. The role... ...applying structured rubrics to evaluate AI outputs with emphasis on workflow, tone... ...capture actionable next steps to train the modeling team. Reliable part-time (10 hours per...Remote jobPart timeFor contractors10 hours per week
- Helpfulness Ranking Reward Model Evaluator is a remote evaluation track for reviewing helpfulness... ...use to retrain. Why this role matters AI data reviewers help turn helpfulness ranking... ...- US-eligible. Remote - Independent specialist contractor. Employment type: CONTRACTOR....Hourly payFor contractorsRemote work10 hours per week
- Table Image Reasoning Model Evaluator is a remote evaluation track for reviewing table image reasoning... ...use to retrain. Why this role matters AI data reviewers help turn table image... ...— US-eligible. Remote · Independent specialist contractor. Employment type: CONTRACTOR....Hourly payFor contractorsRemote work10 hours per week
- AI Trainer Jobs is seeking a remote contractor to evaluate people ops / recruiting prompts and responses against a evolving quality rubric. You will compare model outputs, label edge cases, and provide structured feedback for retraining. Responsibilities include evaluating...Remote jobFor contractors
- RN Annotators is a remote review track for evaluating AI outputs across rn annotators specialist operations workflows. Reviewers grade workflow correctness, policy... ...operational risk; and document the right next step so the modeling team can train on it. RN Annotators is a remote...Remote job
- Fundhouse is seeking an Investment Portfolio Operations Analyst to join our growing team. You will work in our Operations team, supporting the Model Portfolio Service in the UK and collaborating with senior leaders, external providers, and financial advisers. Your role...
$118.8k - $269.8k
Responsibilities About the Team JV AI & Systems Integrity is the... ...and content commitments. We operate the approval gates, testing programs... ...Product Manager for AI Model Assurance owns the roadmap for... ...approval workflow, change detection, evaluation dossier tooling, dashboards,...Temporary workLocal areaShift work$225k - $275k
...Staff Applied AI/ML Engineer, Agents & Model Adaptation Compensation: $225K–... ...accuracy, latency, cost, and operational requirements. Fine-... ...traces into training data and evaluation cases. Build... ...AI People in AI is a specialist search firm focused on AI...Full timePeople In AI
New York, NY1 day ago$140k - $200k
...York, Charlotte, Chicago, and Houston—BIP operates at the intersection of business strategy,... ...industries. Our capabilities span Data & AI, Cybersecurity, Risk & Compliance, Digital... ...Python Engineers to support a high-profile Model Risk transformation program at a leading global...Temporary workRemote workWorldwideBIP US
New York, NY22 hours ago- Agency seeks an experienced voice actor to help shape AI voice models through training data. You will record scripted material, evaluate AI outputs for pronunciation, tone, and pacing, and provide feedback to refine prompts and voice design guidelines. Prior work in voice...Remote jobContract workFreelance
- ...Technology teams constantly improve their ways-of-working to drive productivity and engagement at scale! As a Product Strategy Operating Model Lead - Executive Director, you will lead innovation through the development of strategic initiatives and frameworks spanning an...Full timeWork at officeShift work
- ...Medical Translation AI Evaluator is a remote clinical-review track for evaluating AI outputs... ...corrected clinical reasoning so the modeling team can close the gap. Why this role matters... ...- US-eligible. Remote, Independent specialist contractor. Employment type: CONTRACTOR...Hourly payFor contractorsRemote work10 hours per week
AuraOne, Inc.
New York, NY1 day ago - ...Alignerr Corp. is seeking a Search Quality Evaluator (AI Training) for a fully remote, hourly contract role. You will assess search results, evaluate AI-generated answers, and rate content recommendations to help improve AI performance. The position offers flexible hours...Hourly payContract workRemote workFlexible hours
Alignerr Corp.
New York, NY2 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Model Evaluator & Operational QA Specialist. Be the first to apply!
Related searches
- social media evaluator New York, NY
- web search evaluator New York, NY
- work from home social media evaluator New York, NY
- clinical evaluator New York, NY
- program evaluator New York, NY
- evaluator New York, NY
- ai evaluator New York, NY
- work from home web search evaluator New York, NY
- ads evaluator New York, NY
- education evaluator New York, NY