Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

User Research Expert - Evaluator - AI Trainer

$80 - $120 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: User/customer research and feedback synthesis Evaluator
Type: Contract
Compensation: $80–$120/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated artifacts against domain-specific quality rubrics.
  • Identify factual, aesthetic, and presentation errors.
  • Provide clear, structured written feedback.
  • Work independently and asynchronously to meet deadlines while improving AI model performance .
  • Utilize Microsoft Office and Google Workspace , especially Slides .

Qualifications

Must-Have

  • 5+ years of relevant professional experience in User/customer research and feedback synthesis .
  • Native or professional fluency in English .
  • Highly proficient in Microsoft Office and Google Workspace , especially Slides .

Preferred

  • Advanced degree (Master's or higher) from a reputable institution.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

#hiringmercor
Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the User Research Expert - Evaluator - AI Trainer in San Francisco, CA vacancy
  • $90 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...correct replications from those that merely look correct. Evaluate responsive behavior and semantic quality, ensuring proper use... 
    Suggested
    Contract work
    Summer work
    Local area
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Position: Government / public administration Evaluator Type: Contract Compensation:...  .... Collaborate with subject matter experts to ensure consistency and quality. Work... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Dorsey . Position: Software Engineer Expert Type: Contract Compensation:...  ...: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    26 days ago
  • $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Dorsey . Position: Machine Learning Expert Type: Contract Compensation:...  ...of 8 attempts. Focus on inference, evaluation, and systems work rather than heavy model... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $57 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Responsibilities Write short, realistic requests, similar to what a real user would type. Work with the AI assistant on tasks. Steer it,... 
    Suggested
    Hourly pay
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    18 days ago
  •  ...Join a 16-week research study evaluating AI assistants from the perspective of everyday users. We’re looking for people who already use Gemini, Instinct, or Muse and can assess prompts and responses in English. A part-time commitment of 20 hours per week is available... 
    Full time
    Part time
    Immediate start

    Braintrust

    San Francisco, CA
    1 day ago
  • $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...and Jack Dorsey . Position: Material Science PhD Coding Experts Type: Contract Compensation: $70/hour Location:... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    11 days ago
  • $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Position: General Clinician (MD/DO) - HLS Expert Pool Type: Contract...  ...for consistent model response grading. Evaluate clinical dialogues for accuracy, safety... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    10 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ..., and Jack Dorsey . Position: Healthcare / clinical Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    12 days ago
  • $70 - $110 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...hours/week Role Responsibilities Evaluate AI-generated sales conversations , product...  ..., and post-sale support. Provide expert feedback. Work independently and asynchronously... 
    Hourly pay
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    25 days ago
  • $15 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Dorsey . Position: Music & Lyrics Expert - Malayalam Type: Contract Compensation...  ...hours/week Role Responsibilities Evaluate AI-generated music across various genres... 
    Contract work
    Summer work
    Immediate start
    Remote work
    Flexible hours

    Mercor

    San Francisco, CA
    7 days ago
  • $90 - $140 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...security and responsible AI problems. Evaluate security and responsible AI tasks...  ...Collaborate with other subject matter experts to ensure consistency and accuracy in training... 
    Contract work
    Summer work
    Remote work
    Weekday work

    Mercor

    San Francisco, CA
    8 days ago
  • $84 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Position: Inorganic Chemistry & Synthesis Expert Type: Contract Compensation:...  ...hours/week Role Responsibilities Evaluate technically complex problems involving... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    7 days ago
  • $60 - $70 per hour

     ...and technical talent with leading AI research labs. Headquartered in San Francisco...  ...Dorsey . Position: Application Users - Windows 11 Professional Expert - Professionals Type:...  ...Remote Role Responsibilities Evaluate and utilize Windows 11 Professional... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    4 days ago
  • $85 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Use frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks. Review model-generated... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    1 day ago
  • $60 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...truth outputs and objective rubrics. Evaluate AI models on visual document understanding...  .... Collaborate with subject matter experts to ensure task relevance and accuracy.... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    1 day ago
  • $2,000 per unit

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...and Jack Dorsey . Position: STEM Expert Task Pilot Type: Contract Compensation...  ...AI capabilities. Ensure tasks are evaluated automatically from 0 to 1 with partial... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    6 days ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Dorsey . Position: Sales Engineering Expert Type: Contract Compensation:...  ...objections, supporting proofs of concept, and evaluating product fit and technical risk. ~... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  •  ...We're building a research cohort of senior domain experts across seven specialist tracks — software/systems engineering, formal methods and computational...  ...hardware design, and creative technology — to help evaluate how frontier AI models reason about hard, real-world expert-level... 
    Temporary work

    AfterQuery

    San Francisco, CA
    a month ago
  •  ...Psychology researchers to build the behavioral science scenarios frontier AI labs use to test model judgment. AfterQuery builds the expert evaluation data that frontier AI labs use to measure and improve their models. We're hiring psychology researchers as contractors... 
    Weekly pay
    Contract work
    Temporary work
    For contractors
    Remote work
    10 hours per week

    AfterQuery

    San Francisco, CA
    a month ago
  • $70 - $100 per hour

     ...Open to: USA Join a leading AI lab's cutting-edge GenAI team...  ...Science subject-matter experts (SMEs) from the pharmaceutical...  ...Key Responsibilities Guide research and engineering teams to close...  ...well-structured solutions. Evaluate AI model outputs on drug discovery... 
    Full time
    Contract work
    Weekday work

    Mercor

    San Francisco, CA
    2 days ago
  • $20 per hour

     ...Experience: 1 - 3 years Help improve leading AI models through research, reasoning and educational evaluation. This remote opportunity is for students and working...  ...: AI Evaluator – Education (Experienced Gemini Users) Location: New United States only Apply... 
    Contract work
    Immediate start
    Remote work

    Braintrust

    San Francisco, CA
    21 days ago
  • $127k - $224k

     ...Duties and Responsibilities Own the end-to-end lifecycle of ML models powering product experiences, coverage recommenders, user-to-channel routing, and agent assignment logic Engineer features, prototype experiments, and ship iterative model improvements to... 
    Full time

    Ethos Life

    San Francisco, CA
    3 hours ago
  • $65 - $75 per unit

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Safety , labeled as benign, dual-use, and adversarial. Evaluate model responses against a defined policy standard to ensure correct... 
    Hourly pay
    Weekly pay
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    4 days ago
  • $300k

     ...Staff Data Scientist Grindr is an AI-native platform powering how millions of gay people connect globally. With 15M+ monthly users, 130B+ annual messages, and a team of fewer...  ...broad, open-ended questions Design and evaluate experiments to measure the impact of product... 
    Work at office
    Immediate start
    Worldwide
    Flexible hours

    Grindr

    San Francisco, CA
    1 day ago
  • $145 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco,...  ...Dorsey . Position: ML Research PhD Experts (ICML / NeurIPS / ICLR Publications) Type...  ...: Remote Role Responsibilities Evaluate the accuracy and depth of AI-generated... 
    Remote job
    Contract work
    Summer work

    Mercor

    San Francisco, CA
    9 days ago
  • $150k - $224k

     ...apply advanced statistical modeling, research design, and data science methodologies to evaluate talent programs, organizational...  ...workflows using automation and AI tools. Additionally, you will mentor...  ...world. In addition to our site’s user and privacy policy found at , we... 
    Work experience placement
    Local area

    Indeed

    San Francisco, CA
    1 day ago
  • $150 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include...  ...Summers , and Jack Dorsey . Position: Equity Research Expert — Single-Name Analyst Publications Type: Contract Compensation... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    23 days ago
  •  ...Professional | The Windows 11 Pro desktop itself — Settings, File Explorer, window management and the built-in system tools | Power users, IT administrators, desktop support professionals | Technology Requirement Must be able to capture screenshots with more than 2... 
    Part time
    Immediate start

    Mercor

    San Francisco, CA
    a month ago
  •  ...Sonoma 14.5 | The macOS desktop itself — Finder, System Settings, window and Space management and the bundled applications | Mac power users, Apple-platform administrators, creative and technical professionals | Technology Requirement Must be able to capture... 
    Part time
    Immediate start

    Mercor

    San Francisco, CA
    a month ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to User Research Expert - Evaluator - AI Trainer. Be the first to apply!