AI Evaluation Specialist
Crossing Hurdles
A leading AI organization in Australia is seeking individuals with strong writing and analytical skills to evaluate and improve AI outputs. The ideal candidate must possess the ability to assess emotional nuances and detail while adhering to structured guidelines. Responsibilities include ensuring clarity and empathy in evaluations, identifying weaknesses in AI reasoning, and contributing to a safe AI environment. Immediate availability post-onboarding is required for this role. #J-18808-Ljbffr
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the AI Evaluation Specialist in New York, NY vacancy
$30 per hour
BANGLA AI EVALUATION SPECIALIST Up to $30/hr Depending on the project Bachelor's Remote Part-time Flexible hours Opportunity Overview Handshake is looking for Bangla speakers as it’s spoken in Singapore to help make AI responses more personalized. This is not a traditional...SuggestedPart timeLocal areaRemote workFlexible hours- AuraOne, Inc. seeks a remote contractor to review voice actor CX Agent Voice Cloning Northern UK evaluation outputs. You will label and rate outputs, compare paired responses, and provide structured feedback to retrain the model. Responsibilities include applying rubrics...SuggestedRemote jobFor contractorsFlexible hours
$18 per hour
We are looking for Speech AI Evaluation Specialist to support the improvement of AI-generated content in Chinese Simplified (USA) . Job Type: Freelance Location: USA (work from home) Work Schedule: Part-time - 10+ hours per week. Flexible - work whenever you want. Start...SuggestedHourly payExtra incomePart timeFreelanceImmediate startRemote workWork from home10 hours per weekFlexible hours- ...an Electrical & Electronics Expert for a remote, part-time contract role at $50/hour, supporting 15-20 hours per week. You will evaluate AI models on visual document understanding within the Electrical & Electronics domain and author tasks with clear ground-truth outputs...SuggestedRemote jobContract workPart time
- AI Trainer Jobs seeks a Generalist (Macbook User) to remotely evaluate prompts and responses against AuraOne's rubric. You will compare paired outputs, label edge cases, and draft structured feedback for retraining. Role involves reviewing frontier model outputs, judging...SuggestedRemote job
- Dorado is seeking language professionals for a six‑month, independent contractor engagement to contribute to an AI evaluation project. Review AI‑generated content, annotate language data, and provide feedback to improve accuracy and reliability. The role requires reliability...For contractors
- AI Evaluation Specialist Role Type: Contractor Location: Remote — US, Canada, UK, Ireland, Australia, New Zealand We are seeking experienced AI Evaluation Specialists to assess and improve the quality of AI assistant outputs for an enterprise AI training project. You'll...Remote jobFor contractors
- Ophthalmology AI Evaluation Specialist — Clinical Reasoning Review is a remote evaluation track for reviewing ophthalmology ai evaluation evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the...Hourly payFor contractorsRemote work10 hours per week
- Anyone AI is seeking remote independent contractors to help create and evaluate data for frontier AI models. You will interact with AI models and compare responses to ensure quality and reliability. Responsibilities include assessing accuracy, helpfulness, and instruction...Remote jobFor contractors
- AuraOne, Inc. is seeking an independent contractor to review Portuguese generalist evaluation outputs. You will apply a versioned rubric to each pair of model responses, assign severity tags, and provide structured rationales for the best answer. The role emphasizes high...Remote jobFor contractorsFlexible hours
- AI Trainer Jobs is seeking contributors for remote language evaluation work. You will assess Slovak translations and localization, rate model outputs, and provide corrected answers to improve AI training. The role offers flexible hours, remote collaboration, and weekly...Remote jobFor contractorsFlexible hours
$197.3k - $225.1k
AI Engineer 4 (AI Foundations: Benchmarking, Evaluation, and Explainability) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been an industry leader in using machine learning to create real-time,...Full timePart timeLocal areaCapital One
New York, NY2 days ago- ...Job TitleAI Evaluation EngineerLocationHybrid / RemoteEmployment TypeFull-timeJob SummaryWe are seeking an AI Evaluation Engineer to design, implement, and maintain evaluation frameworks for AI and machine learning systems, with a focus on Large Language Models (LLMs)...
Ova Technologies
New York, NY20 hours ago - e2f Inc. is seeking an AI Research & Evaluation Analyst for remote, project-based work. You will support research and rubric development for advanced AI and large language model projects, requiring strong English writing, careful reasoning, and reliable sourcing. Responsibilities...Remote job
- Mercor is seeking a remote IP/Trademark/Copyright Law Evaluator to assess AI-generated artifacts against strict quality rubrics. You will identify factual and presentation errors and provide structured feedback to improve AI model performance. This contract role requires...Remote jobContract work
$20 - $80 per hour
Role Overview Help improve next-generation AI systems by supplying precise, real-world evaluation, annotation, and feedback. This remote contractor role focuses on how AI models learn, reason, and perform across diverse subject areas. Key Responsibilities Evaluate and...Hourly payContract workFor contractorsRemote work$70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Contract Compensation: $70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses to ensure accuracy and depth in reasoning...Contract workSummer workRemote work$40 per hour
A leading company in AI training solutions is seeking a Content Developer specialized in Physics. The role involves enhancing AI models by providing and evaluating complex physics problems. Ideal candidates hold advanced degrees in physics or related fields, ensuring high...Remote jobHourly pay- Volga Partners is seeking an experienced Quantitative Finance Subject Matter Expert to support the evaluation and improvement of AI-driven financial analyses in a remote, project-based capacity. You will review AI-generated models, identify inaccuracies, and provide structured...Remote work
$11 - $30.65 per hour
Meridial is seeking contractors to evaluate advanced agentic audio models by simulating realistic customer service interactions across multiple domains. You will contribute to developing diverse datasets and assess model performance using various metrics. The role requires...Remote jobHourly payFor contractors$30 - $60 per hour
Jobgether SRL in the United States seeks a remote, contract-based role to design realistic medical tasks for evaluating AI in healthcare. This part-time position offers about 20 hours per week and remote work flexibility. You will create tasks involving clinical analysis...Remote jobHourly payContract workPart time- ...About Turing: Turing is one of the world’s fastest-growing AI companies accelerating the advancement and deployment of powerful AI... ...detail-oriented Business Analyst to support a project involving the evaluation of multiple chatbot responses. The role will require analyzing...FreelanceRemote workFlexible hoursWeekday work
Braintrust
New York, NY6 hours ago $15 - $30 per hour
...Contractor (Remote) to perform realistic, everyday digital tasks using AI assistant tools under standardized conditions. Compensation is $... ..., verify results for accuracy, apply grading guidelines to evaluate responses, and test scenarios involving email, calendar invites,...Remote jobHourly payFor contractors10 hours per week- Zingage (Kuzushi Inc.) is building the intelligence layer for home-based care in New York City. We seek a founding AI engineer to own evaluation, simulation, retrieval, and alignment infrastructure for autonomous agents in a high-impact healthcare setting. You will ship...
$20 per hour
Prolific is seeking a Sales Professional to join our Expert Network for AI model training and evaluation. You will be paid to contribute real sales and business development expertise via paid tasks, with a quick screening test if selected. All work is remote and ad hoc...Remote jobHourly payFreelance$20 per hour
Prolific is seeking Sales Professionals to join its Expert Network for AI training tasks. You’ll apply real sales and business development experience to help train and evaluate models. A quick 10-15 minute test is required, and successful candidates join the Prolific participant...Remote jobHourly payFreelance- Scale Labs seeks a Research Scientist focused on Frontier Risk Evaluations to design evaluation measures and harnesses for frontier AI systems. You will build datasets, test AI models, and collaborate with government agencies to scope and design risk evaluations. Ideal...
- SaidGig is seeking a research-focused professional to advance trustworthy AI for healthcare by designing evaluation frameworks for medical reasoning, evidence synthesis, and decision support. This full-time remote role blends clinical insight with AI benchmarking and standards...Remote jobFull time
$400k
24-MAG LLC is seeking experienced medical and healthcare researchers to design rigorous evaluation frameworks for trustworthy healthcare-focused AI systems. You will develop benchmark datasets, validation environments, and analyse medical reasoning to improve AI reliability...Remote jobFull time- Crossing Hurdles in the United Kingdom seeks a finance and AI governance specialist to develop realistic, multi-step finance and business process prompts mirroring real industry workflows. You will create objective rubrics to assess AI outputs, review responses using these...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluation Specialist. Be the first to apply!
