Analytical Evaluator - AI Feedback
$70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
- Ensure consistent and honest judgment, including critical assessments when warranted.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-500 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Native English fluency required.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
- ...Join a 16-week research study evaluating AI assistants from the perspective of everyday users. We’re looking for people who already use... ...as an everyday user. Assess response quality and provide feedback following the study’s guidelines. Participate consistently...SuggestedFull timePart timeImmediate start
- Mercor is seeking a Procurement / Vendor Management Evaluator for a remote contract position. The role focuses on assessing AI-generated artifacts, identifying errors, and providing structured feedback to improve AI model outputs. You will collaborate with AI research teams...SuggestedRemote jobContract work
$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Incident management / reliability / SRE Evaluator Type: Contract Compensation: $... ...decks. Provide clear, structured written feedback to improve AI-generated work products....SuggestedContract workSummer workWork at officeRemote work$70 - $110 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...+ hours/week Role Responsibilities Evaluate AI-generated operational plans , staff... ...performance improvement decisions. Provide expert feedback. Work independently and...SuggestedHourly payContract workSummer workImmediate startRemote work$60 - $70 per hour
...technical talent with leading AI research labs. Headquartered... ...Role Responsibilities Evaluate AI-generated responses for... ...violations. Provide structured feedback to improve model alignment and... ..., critical thinking, and analytical reasoning skills. ~ Ability...SuggestedContract workSummer workRemote work- Mercor is seeking an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance, covering... ...discipline. Excellent English writing and analytic skills are required; flexible remote work across...Remote jobContract workFlexible hours
- ...Francisco, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. As a Software Engineering evaluator, you will curate code examples, provide precise solutions, and refine AI-generated...For contractors
$20 per hour
...Experience: 1 - 3 years Help improve leading AI models through research, reasoning and educational evaluation. This remote opportunity is for students and... ...~1+ year of experience, with strong research, analytical and written communication skills. ~ Fluency in...Contract workImmediate startRemote work- ...About the role AI assistants are becoming part of everyday life, helping people with... ...everyday conversations, and share your feedback on what worked, what didn't, and which experience... ...and select your preferred experience. Evaluate conversational quality, helpfulness,...Hourly payTemporary workPart timeRemote work
$50 - $150 per hour
...hiring PhD-level quantitative finance experts to help train and evaluate advanced AI models on financial and quantitative reasoning. This is... ..., risk modeling, or computational finance Strong analytical and critical-thinking skills, with the ability to spot subtle...Contract workTemporary workRemote workFlexible hours$10 - $30 per hour
...communication attention to detail constructive feedback digital delivery platforms ai-driven tools familiarity independent work adaptability... ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve...Part timeFor contractorsRemote workShift work$20 - $160 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $... ...Preferred Experience in annotation, human-feedback, evaluation, or content review....Contract workSummer workRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $... ...errors. Provide clear, structured written feedback. Collaborate with subject matter experts...Contract workSummer workWork at officeRemote work$30 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ..., answering questions, and correcting errors. Evaluate the experience and deliverables, providing concise feedback on performance. Qualifications Must-Have...Hourly payWeekly payContract workFor contractorsSummer workRemote work$25 - $45 per hour
...Required Skills ChatGPT Muse AI agents Attention to Detail Following... ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments... ..., and time Share clear written feedback on the agent experience Who we’re looking...Part timeFor contractorsWork at officeWeekend work$90 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...genuinely correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use of...Contract workSummer workLocal areaRemote work$85 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-...Contract workSummer workRemote work- ...Luxury Brand Evaluator CXG is a global Customer Experience consulting firm specializing exclusively in serving premium and luxury... ...to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques...WorldwideFlexible hours
$290k - $365k
...create reliable, interpretable, and steerable AI systems. We want AI to be safe and... ...Anthropic gives independent external evaluators access to our models, our people, and our... ...on how to work with evaluators Collect feedback from evaluators and internal teams, run retrospectives...Contract workWork at officeLocal areaVisa sponsorshipFlexible hours- ...be a great match for you. Become a Luxury Automotive Experience Evaluator As a Luxury Automotive Experience Evaluator, you'll be invited to visit premium automotive showrooms and provide expert feedback on the customer journey. Your perspective, shaped by your familiarity...FreelanceFlexible hours
$120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Procurement / vendor management Evaluator Type: Contract Compensation: $80-$120/hour... ...decks. Provide clear, structured written feedback to improve AI model outputs . Collaborate...Remote jobContract workSummer workWork at office- A leading luxury brand evaluator is seeking a Luxury Automotive Experience Evaluator in San Francisco. This freelance position offers... ...brand specialists, evaluate service quality, and provide unbiased feedback. Ideal candidates are passionate about automobiles, have a...FreelanceFlexible hours
$30 per hour
...Review real user interaction traces with an AI shopping assistant Identify logical... ...quality Commit to 20+ hours per week of evaluation work on our internal platform Who Should... ...assurance, or AI training Strong analytical skills with the ability to spot subtle errors...Contract workFor contractorsRemote work$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in... .... Provide clear, structured written feedback to improve AI model outputs . Apply...Contract workSummer workWork at officeRemote work- Mercor is seeking a Biology / environmental science Evaluator for a remote contract role. You will assess AI-generated artifacts using domain-specific rubrics, identify errors, and provide structured feedback to improve content quality. Candidates should have 5+ years...Remote jobContract workWork at office
$70 per hour
...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers...Contract workSummer workImmediate startRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Biology / environmental science Evaluator Type: Contract Compensation: $80-$120/hour... ...outputs. Provide clear, structured written feedback to improve AI-generated content. Review documents...Contract workSummer workWork at officeRemote work$70 - $110 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...+ hours/week Role Responsibilities Evaluate AI-generated legal memoranda , contracts... ...client advisory decisions, providing expert feedback. Collaborate with AI research teams...Hourly payContract workSummer workImmediate startRemote work$180 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...5 hours/week Role Responsibilities Evaluate AI-generated legal responses to enhance... ..., and debt law . Provide structured feedback to improve AI systems . Apply experience...Contract workSummer workRemote work$150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...discrete, checkable criteria for consistent model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and...Contract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!
- evaluator San Francisco, CA
- work from home web search evaluator San Francisco, CA
- ai evaluator San Francisco, CA
- education evaluator San Francisco, CA
- quality evaluator San Francisco, CA
- social media evaluator San Francisco, CA
- vocational evaluator
- evaluator
- admissions evaluator
- work from home web search evaluator



