Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Analytical Evaluator - AI Feedback

$70 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
  • Provide structured written feedback to improve training data quality and downstream performance.
  • Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
  • Ensure consistent and honest judgment, including critical assessments when warranted.
  • Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

Must-Have

  • Bachelor's degree from a top-500 globally ranked university preferred.
  • Strong analytical and written communication skills.
  • Ability to work independently and follow detailed task guidelines.
  • Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
  • Native English fluency required.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 27 days ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback in San Francisco, CA vacancy
  •  ...Join a 16-week research study evaluating AI assistants from the perspective of everyday users. We’re looking for people who already use...  ...as an everyday user. Assess response quality and provide feedback following the study’s guidelines. Participate consistently... 
    Suggested
    Full time
    Part time
    Immediate start

    Braintrust

    San Francisco, CA
    1 day ago
  • Mercor is seeking a Procurement / Vendor Management Evaluator for a remote contract position. The role focuses on assessing AI-generated artifacts, identifying errors, and providing structured feedback to improve AI model outputs. You will collaborate with AI research teams... 
    Suggested
    Remote job
    Contract work

    Mercor

    San Francisco, CA
    5 days ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Incident management / reliability / SRE Evaluator Type: Contract Compensation: $...  ...decks. Provide clear, structured written feedback to improve AI-generated work products.... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $70 - $110 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...+ hours/week Role Responsibilities Evaluate AI-generated operational plans , staff...  ...performance improvement decisions. Provide expert feedback. Work independently and... 
    Suggested
    Hourly pay
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    27 days ago
  • $60 - $70 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Role Responsibilities Evaluate AI-generated responses for...  ...violations. Provide structured feedback to improve model alignment and...  ..., critical thinking, and analytical reasoning skills. ~ Ability... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  • Mercor is seeking an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance, covering...  ...discipline. Excellent English writing and analytic skills are required; flexible remote work across... 
    Remote job
    Contract work
    Flexible hours

    Mercor

    San Francisco, CA
    3 days ago
  •  ...Francisco, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. As a Software Engineering evaluator, you will curate code examples, provide precise solutions, and refine AI-generated... 
    For contractors

    Parailabs

    San Francisco, CA
    4 days ago
  • $20 per hour

     ...Experience: 1 - 3 years Help improve leading AI models through research, reasoning and educational evaluation. This remote opportunity is for students and...  ...~1+ year of experience, with strong research, analytical and written communication skills. ~ Fluency in... 
    Contract work
    Immediate start
    Remote work

    Braintrust

    San Francisco, CA
    21 days ago
  •  ...About the role AI assistants are becoming part of everyday life, helping people with...  ...everyday conversations, and share your feedback on what worked, what didn't, and which experience...  ...and select your preferred experience. Evaluate conversational quality, helpfulness,... 
    Hourly pay
    Temporary work
    Part time
    Remote work

    micro1

    San Francisco, CA
    4 days ago
  • $50 - $150 per hour

     ...hiring PhD-level quantitative finance experts to help train and evaluate advanced AI models on financial and quantitative reasoning. This is...  ..., risk modeling, or computational finance Strong analytical and critical-thinking skills, with the ability to spot subtle... 
    Contract work
    Temporary work
    Remote work
    Flexible hours

    Braintrust

    San Francisco, CA
    a month ago
  • $10 - $30 per hour

     ...communication attention to detail constructive feedback digital delivery platforms ai-driven tools familiarity independent work adaptability...  ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments to improve... 
    Part time
    For contractors
    Remote work
    Shift work

    micro1

    San Francisco, CA
    21 days ago
  • $20 - $160 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $...  ...Preferred Experience in annotation, human-feedback, evaluation, or content review.... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $...  ...errors. Provide clear, structured written feedback. Collaborate with subject matter experts... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  • $30 per hour

     ...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ..., answering questions, and correcting errors. Evaluate the experience and deliverables, providing concise feedback on performance. Qualifications Must-Have... 
    Hourly pay
    Weekly pay
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    18 days ago
  • $25 - $45 per hour

     ...Required Skills ChatGPT Muse AI agents Attention to Detail Following...  ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments...  ..., and time Share clear written feedback on the agent experience Who we’re looking... 
    Part time
    For contractors
    Work at office
    Weekend work

    micro1

    San Francisco, CA
    13 days ago
  • $90 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...genuinely correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use of... 
    Contract work
    Summer work
    Local area
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  • $85 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Responsibilities Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    4 days ago
  •  ...Luxury Brand Evaluator CXG is a global Customer Experience consulting firm specializing exclusively in serving premium and luxury...  ...to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques... 
    Worldwide
    Flexible hours

    CXG

    San Francisco, CA
    4 days ago
  • $290k - $365k

     ...create reliable, interpretable, and steerable AI systems. We want AI to be safe and...  ...Anthropic gives independent external evaluators access to our models, our people, and our...  ...on how to work with evaluators Collect feedback from evaluators and internal teams, run retrospectives... 
    Contract work
    Work at office
    Local area
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    23 hours ago
  •  ...be a great match for you. Become a Luxury Automotive Experience Evaluator As a Luxury Automotive Experience Evaluator, you'll be invited to visit premium automotive showrooms and provide expert feedback on the customer journey. Your perspective, shaped by your familiarity... 
    Freelance
    Flexible hours

    CXG

    San Francisco, CA
    5 days ago
  • $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Procurement / vendor management Evaluator Type: Contract Compensation: $80-$120/hour...  ...decks. Provide clear, structured written feedback to improve AI model outputs . Collaborate... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    San Francisco, CA
    5 days ago
  • A leading luxury brand evaluator is seeking a Luxury Automotive Experience Evaluator in San Francisco. This freelance position offers...  ...brand specialists, evaluate service quality, and provide unbiased feedback. Ideal candidates are passionate about automobiles, have a... 
    Freelance
    Flexible hours

    CXG

    San Francisco, CA
    3 days ago
  • $30 per hour

     ...Review real user interaction traces with an AI shopping assistant Identify logical...  ...quality Commit to 20+ hours per week of evaluation work on our internal platform Who Should...  ...assurance, or AI training Strong analytical skills with the ability to spot subtle errors... 
    Contract work
    For contractors
    Remote work

    Terac

    San Francisco, CA
    a month ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in...  .... Provide clear, structured written feedback to improve AI model outputs . Apply... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    12 days ago
  • Mercor is seeking a Biology / environmental science Evaluator for a remote contract role. You will assess AI-generated artifacts using domain-specific rubrics, identify errors, and provide structured feedback to improve content quality. Candidates should have 5+ years... 
    Remote job
    Contract work
    Work at office

    Mercor

    San Francisco, CA
    4 days ago
  • $70 per hour

     ...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    23 days ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Biology / environmental science Evaluator Type: Contract Compensation: $80-$120/hour...  ...outputs. Provide clear, structured written feedback to improve AI-generated content. Review documents... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    4 days ago
  • $70 - $110 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...+ hours/week Role Responsibilities Evaluate AI-generated legal memoranda , contracts...  ...client advisory decisions, providing expert feedback. Collaborate with AI research teams... 
    Hourly pay
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    11 days ago
  • $180 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...5 hours/week Role Responsibilities Evaluate AI-generated legal responses to enhance...  ..., and debt law . Provide structured feedback to improve AI systems . Apply experience... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  • $150 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...discrete, checkable criteria for consistent model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    10 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!