Expert Talent Pool, Evals and Benchmarks (Associate)
$30 - $60 per hourClera
ABOUT THE ROLE Join a specialist talent pool supporting human-data projects that evaluate AI models. You will help design and run assessments that measure model capabilities and contribute to AI safety and capability research. WHAT YOU'LL DO - Write questions and tasks that isolate and test specific model capabilities. - Develop rubrics that define strong, partial, and poor answers. - Create reference answers and edge cases to challenge AI models. - Grade model outputs against rubrics and explain scoring decisions. - Analyze failed answers, group error patterns, and document findings. - Compare multiple models using identical prompts. - Validate benchmarks and flag compromised or ineffective questions. - Participate in small user studies evaluating ratings or interactions with model outputs.
WHAT WE'RE LOOKING FOR - Zero to five years of professional experience. - Experience in QA or test engineering, assessment design, psychometrics, UX research, data analysis, ML evaluation, research support, or relevant subject-matter work involving question writing. - Experience writing test cases, exam questions, rubrics, or evaluation tasks, and reviewing work consistently against written criteria. - Comfort using spreadsheets or Python to organize and analyze results, and identifying and documenting error patterns. - Experience with evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo is useful. - Clear written communication and the ability to compare AI model outputs or participate in user and rating studies. COMPENSATION & BENEFITS Compensation is $30 to $60 per hour, depending on project, field, and experience. Payment is for hours worked. LOCATION Fully remote. Projects typically last from one week to several months.
$75 - $175 per hour
...ABOUT THE ROLE Join a network of senior evaluation experts supporting AI research projects. You will lead benchmark design, validation, and grading operations when projects align with your expertise, helping produce rigorous and reliable assessments of AI models. WHAT...SuggestedHourly payRemote work$62.4k - $124.8k
...About the Role Join an expert network supporting AI model evaluation projects as a safety tester. You will identify and document ways AI systems can fail, helping improve model safety across projects matched to your background. What You'll Do Write prompts and...SuggestedHourly payFull timeRemote work$25 - $50 per hour
...ABOUT THE ROLE Join a flexible, project-based expert talent pool contributing to AI training work. You will write, edit, and evaluate examples that help AI models reason and respond effectively. WHAT YOU'LL DO - Write prompts and ideal responses for conversational AI...SuggestedHourly payRemote workFlexible hours$35 - $70 per hour
...ABOUT THE ROLE Join a paid talent pool supporting projects in AI agents and reinforcement learning. You will help build practice environments and evaluate agent behavior, supporting the development of AI systems that interact with software and workflows. WHAT YOU'LL DO...SuggestedHourly payRemote work$20 - $45 per hour
...ABOUT THE ROLE Join a project-based expert network supporting AI model evaluation and improvement. You will provide structured feedback that helps make model responses more accurate, helpful, and safe. WHAT YOU'LL DO - Rank and compare model responses to identify the...SuggestedHourly payRemote work$15 - $35 per hour
...ABOUT THE ROLE Join an expert network supporting AI training projects as an annotation specialist. You will label diverse data types to help improve how AI systems learn, with project assignments matched to your background. WHAT YOU'LL DO - Classify images, video, audio...Hourly payRemote work$17 - $22 per hour
...Associate Patient Access Specialist Help patients get the care they need with patience, clarity, and compassion. As an Associate Patient... ...same hours, this job won't be the best match. About Our Talent Pool Hummingbird is growing fast, and we interview year-round...Hourly payFull timeWork experience placementRemote workMonday to Friday- Talent Pool: Financial Well-being Expert Remote Are you interested in joining us as a Freelance Financial Well-being expert, but don't see a specific role currently listed on our website? We are always looking to connect with talented professionals, so we've created a...Permanent employmentContract workFreelanceLocal areaRemote work
- ...Job Description Job Description Are you a talented professional with experience in medical devices , diagnostics , or biotech ?... ...of advancing health through innovation. By joining our Talent Pool , you’ll be among the first we contact when new roles open up in...Work at officeRemote work1 day per week
$83.2k - $156k
...About the Role Join an expert annotation network supporting advanced AI research projects. You will work on complex, multimodal data labeling and help maintain high quality across projects. What You'll Do Label difficult or ambiguous examples across images, video...Hourly payFull timeRemote work$61 - $77 per hour
...Mercor connects elite creative and technical talent with leading AI research labs.... ...in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel ,... ...question's difficulty as Medium, Hard, or Expert, providing 1 correct answer and 9 plausible...Contract workSummer workRemote work$61 - $77 per hour
...Mercor connects elite creative and technical talent with leading AI research labs.... ...in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel ,... ...undergraduate), Hard (advanced undergraduate), or Expert (post-graduate and above). Provide 1...Contract workSummer workRemote work$150 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered... ...San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel ,... ...Position: General Clinician (MD/DO) - HLS Expert Pool Type: Contract Compensation:...Contract workSummer workRemote work- ...About this opportunity This is a talent pool application, not a role with an immediate start date. Our open pools cover the areas... ...produced by other professionals against those rubrics. This is expert, task-based work performed remotely and asynchronously. What...Part timeImmediate startRemote work
- ...About this opportunity This is a talent pool application, not a role with an immediate start date. We are building a vetted network... ...other legal professionals against those rubrics. This is expert, task-based work performed remotely and asynchronously. Requirements...Part timeImmediate startRemote work
$61 - $77 per hour
...Mercor connects elite creative and technical talent with leading AI research labs.... ...in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel ,... ...question's difficulty as Medium, Hard, or Expert, providing 1 correct answer and 9 plausible...Contract workSummer workRemote work$100 - $140 per hour
...improve how well AI systems handle real questions of law across multiple jurisdictions. We keep an ongoing talent pool of qualified, practising lawyers who serve as expert reviewers on these projects. This is a rolling listing: we review applications on a continuous basis...Contract workRemote workFlexible hours$100 - $140 per hour
...Lawyers & Legal Experts — Global Talent Pool (UK, Germany, France, Spain, Australia... is a remote review track for evaluating AI outputs in legal... ...policy reviewers to grade AI outputs the way a senior associate would — with citations, severity tags, and the corrected...Remote jobFor contractors10 hours per week- ...Associate Vice President, Talent About the Company Highly rated community bank offering personalized financial services in North Alabama... ...hiring, maintains job descriptions, coordinates compensation benchmarking and incentive program design, and analyzes workforce...Work at office
$160 per hour
...Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack...Remote jobContract workSummer work- ...Project: Life Sciences Research & Benchmarking Study Role: Subject Matter Expert (SME) – Ecological, Evolutionary & Environmental Sciences Qualification: PhD in a relevant Life Sciences discipline Engagement: Project-Based / Research Assignment About the...Temporary work
- ...Position: Life Sciences Research & Benchmarking Expert Qualification: PhD in a relevant Life Sciences discipline Engagement: Project-Based / Research Evaluation About the Project We are seeking highly qualified PhD professionals in Life Sciences with strong...Temporary work
$45 - $90 per hour
...planning, policy and employee lifecycle, that test AI systems. You also judge how well models handle them. What you will do Create HR benchmark tasks with expected answers Review AI outputs on HR policy and operations Write clear rationales Keep tasks realistic and current...Hourly payWeekly payContract workPart timeFor contractorsRemote workFlexible hours$71.93k
Online Talent Development Expert Recruitment Firm: Naukri Mitra Job Title: Online Talent Development Expert Location: Remote Annual Salary: $71,927 About the Role: Naukri Mitra is excited to announce an opportunity for an experienced and innovative Online Talent Development...Remote workWorldwide- ...personalized outreach to senior professionals. You will identify suitable candidates on LinkedIn, XING and company sites, build talent pools, and coordinate interviews while leveraging AI tools like ChatGPT to improve search methods and messaging. Fluency in German (C1...Remote job
- Prolific Academic Ltd invites HR and Talent Acquisition professionals to join its Expert Network to train and evaluate AI models with real-world HR expertise. You will assess AI-generated HR content, create realistic HR scenarios, and annotate data to improve model accuracy...Remote jobWork from homeFlexible hours
- ...Tuesdays, Wednesdays and a third day unique to each team or employee). The Impact you will have in this role: Being a member of the Talent Acquisition team, you will be a steward of DTCC’s greatest asset - our people. The Sr Sourcer partners closely with Recruiters to...Full timeWork at officeRemote workFlexible hours
- Rockstar is a US-based remote staffing company seeking a Talent Acquisition Specialist with hands-on medical billing experience to join our internal team. You will source and screen billing professionals, build a steady candidate pipeline, and support partner practices...Remote job
- Prolific is seeking HR and Talent Professionals in San Jose, California to join their Expert Network. Responsibilities include evaluating AI-generated content and creating HR scenarios. Ideal candidates will have professional HR experience, strong compliance expertise,...Remote jobHourly payWork from homeFlexible hours
- Full-time remote role blending medical billing expertise with talent acquisition. Supports U.S. therapy practices by hiring skilled RCM professionals. Key Responsibilities: Source and screen candidates using hands-on billing knowledge. Assess proficiency via live calls...Remote jobFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Expert Talent Pool, Evals and Benchmarks (Associate). Be the first to apply!





