Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Analytical Evaluator - AI Feedback

$70 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote

Role Responsibilities

  • Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
  • Provide structured written feedback to improve training data quality and downstream performance.
  • Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
  • Ensure consistent and honest judgment, including critical assessments when warranted.
  • Work independently and asynchronously to meet deadlines while improving AI model performance .

Qualifications

Must-Have

  • Bachelor's degree from a top-500 globally ranked university preferred.
  • Strong analytical and written communication skills.
  • Ability to work independently and follow detailed task guidelines.
  • Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
  • Native English fluency required.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback in New York, NY vacancy
  •  ...Description Job Title: Food Delivery AI Response Evaluator Role Type: Contractor Location:...  ...effectiveness. Your industry knowledge and feedback will help improve AI systems. Prior AI...  ...tools is an advantage. # Strong analytical skills and the ability to provide clear... 
    Suggested
    Remote job
    For contractors

    YO AI Labs

    New York, NY
    11 days ago
  • $70 - $110 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Role Responsibilities Evaluate AI-generated financial plans...  ..., regulatory compliance, and analytical rigor. Review financial scenarios...  ...reporting. Provide expert feedback to improve AI model... 
    Suggested
    Hourly pay
    Contract work
    Summer work
    Work at office
    Immediate start
    Remote work

    Mercor

    New York, NY
    1 day ago
  • $30 per hour

     ...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ..., answering questions, and correcting errors. Evaluate the experience and deliverables, providing concise feedback on performance. Qualifications Must-Have... 
    Suggested
    Remote job
    Hourly pay
    Weekly pay
    Contract work
    For contractors
    Summer work

    Mercor

    New York, NY
    22 days ago
  • $40 - $80 per hour

     ...Description Job Description Job Description: Paralegal – AI Training & Legal Evaluation Role Type: Contractor Location: Remote (Worldwide,...  .... # Familiarity with AI or LLM evaluation. # Strong analytical, organizational, and problem-solving skills.... 
    Suggested
    Remote job
    Hourly pay
    For contractors
    Worldwide
    10 hours per week
    Flexible hours

    YO AI Labs

    New York, NY
    6 days ago
  • $90 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Location: Remote Role Responsibilities Evaluate the quality and correctness of AI-...  .... Provide clear, rubric-based written feedback to improve AI model outputs . Review... 
    Suggested
    Remote job
    Contract work
    Summer work

    Mercor

    New York, NY
    16 days ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...regulatory response with financial-services AI Evaluator Type: Contract Compensation: $...  .... Provide clear, structured written feedback to improve AI model outputs. Apply deep... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    15 days ago
  • $137.9k - $256.1k

    Sr. Legal AI Prompter / EvaluatorJob DescriptionSr. Legal AI Prompters / Evaluators are responsible for improving CoCounsel Legal by writing and refining prompts, skills...  ...retrieval, or content), and provide strategic feedback to software engineers and data... 
    Full time
    Work at office
    Local area
    Flexible hours

    Thomson Reuters

    New York, NY
    16 hours ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $...  ...outputs. Provide clear, structured written feedback to improve quality. Work independently... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    21 days ago
  • $70 - $110 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...+ hours/week Role Responsibilities Evaluate AI-generated operational plans , staff...  ...and workflow coordination. Provide expert feedback. Leverage real-world expertise in directing... 
    Hourly pay
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    New York, NY
    16 days ago
  • $60 - $70 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Role Responsibilities Evaluate AI-generated responses for...  ...violations. Provide structured feedback to improve model alignment and...  ..., critical thinking, and analytical reasoning skills. ~ Ability... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    15 days ago
  • $20 - $160 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $...  ...Preferred Experience in annotation, human-feedback, evaluation, or content review.... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    4 days ago
  • $80 - $120 per hour

     ...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...and Jack Dorsey . Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    15 days ago
  • $80 - $120 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Biology / environmental science Evaluator Type: Contract Compensation: $...  ...outputs. Provide clear, structured written feedback to improve AI-generated content.... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    4 days ago
  • YO AI Labs is seeking experienced Humanitarian Aid Representatives to contribute expertise in humanitarian operations, civilian protection...  ...and International Humanitarian Law (IHL) to an AI training and evaluation project. You will help evaluate and refine AI-generated... 
    Remote job

    YO AI Labs

    New York, NY
    1 day ago
  • $70 per hour

     ...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    New York, NY
    28 days ago
  • $160 per hour

     ...and technical talent with leading AI research labs. Headquartered in...  ...relevance. Provide structured feedback to improve document standards and evaluation processes. Utilize Excel ,...  ...in a relevant domain ~ Strong analytical and written communication skills... 
    Remote job
    Contract work
    Summer work

    Mercor

    New York, NY
    a month ago
  • $190 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Remote Commitment: 20+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows, including personal... 
    Remote job
    Hourly pay
    Contract work
    For contractors
    Summer work
    Trial period

    Mercor

    New York, NY
    15 days ago
  • $50 per hour

     ...Prolific Academic Ltd is seeking Product Designers and UX Specialists to join our Expert Network to train and evaluate AI models using real design expertise. If selected, you’ll complete a brief test and, upon success, join Prolific as a paid participant to work on AI... 
    Hourly pay
    Remote work
    Work from home
    Flexible hours

    Prolific Academic Ltd

    New York, NY
    3 days ago
  •  ...Description Job Title: Language Model Evaluator Job Summary We are seeking a...  ...Models (LLMs) and Generative AI applications. The ideal...  ..., and test cases. Strong analytical skills with the ability to...  ...reinforcement learning from human feedback (RLHF) concepts.... 
    Full time
    Remote work

    Ova Technologies

    New York, NY
    26 days ago
  • $100 - $120 per unit

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...and Jack Dorsey . Position: Baseball Fan – Live MLB Game AI Evaluator (US) Type: Contract Compensation: $100–$120 per task... 
    Contract work
    Work experience placement
    Summer work
    Remote work

    Mercor

    New York, NY
    18 days ago
  • $80 - $150 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Position: MS Excel / Google Sheets Evaluator Type: Contract...  ...relevance. Provide structured feedback to improve document...  ...and Google Slides ~ Strong analytical and written communication skills... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    16 days ago
  • $50 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Role Responsibilities Evaluate the quality and accuracy of...  ...and language gaps, providing feedback to improve LLM performance...  ...Strong attention to detail, analytical judgment, and written communication... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    16 days ago
  •  ...Job Description Job Description Role Title: AI Agent Evaluator Role Type: Contractor Location: United States We are seeking detail...  ...agents side by side, assess their performance, and provide feedback to improve their reliability and user experience. Key... 
    Remote job
    For contractors
    Work at office

    YO AI Labs

    New York, NY
    11 days ago
  • YO AI Labs is seeking experienced physicians to support an advanced healthcare AI project. This part-time contractor role involves applying clinical expertise to evaluate AI-generated medical content, analyze cases, and improve AI systems used in healthcare decision-making... 
    Remote job
    Part time
    For contractors

    YO AI Labs

    New York, NY
    2 days ago
  • $60k - $100k

     ...mindset—eager to learn, adapt, and evolve with the company Ability to work independently while remaining open to feedback and process improvements Strong analytical skills and attention to detail Proficiency with appraisal software and market research tools A professional... 
    For contractors

    Banks Valuation

    New York, NY
    3 days ago
  • $12 per hour

    Turkish AI Product Evaluator Productive Playhouse is building a talent pool of Turkish speakers for an upcoming project testing and evaluating leading AI chatbots. The objective is to enhance response quality through direct user interaction with the various AI models.... 
    Hourly pay
    For contractors
    Freelance
    Remote work
    Worldwide
    Flexible hours

    Productive Playhouse

    New York, NY
    3 days ago
  •  ...Web Browsing Evaluator Contractor Remote Micro1 is engaging Web Browsing Evaluators...  ...expertise to help train next-generation AI systems. Your work will shape how models...  ...Provide clear written documentation and feedback on the browsing process, highlighting any... 
    For contractors
    Remote work

    ESR Healthcare

    New York, NY
    4 days ago
  •  ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Commitment: Flexible part-time hours Role Responsibilities Evaluate AI answers to endocrinology questions for clinical accuracy,... 
    Contract work
    Part time
    Summer work
    Immediate start
    Remote work
    Flexible hours

    Mercor

    New York, NY
    a month ago
  • Luxury Brand Evaluator Turn your passion for luxury into a career opportunity! Explore the world of premium brands and make a lasting...  ...to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques... 
    Worldwide

    CXG

    New York, NY
    3 days ago
  • $80 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...  ...ETL pipelines , data warehouses , analytics platforms , and distributed data systems... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    18 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!