Analytical Evaluator - AI Feedback
$70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
- Ensure consistent and honest judgment, including critical assessments when warranted.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-500 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Native English fluency required.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
- ...Description Job Title: Food Delivery AI Response Evaluator Role Type: Contractor Location:... ...effectiveness. Your industry knowledge and feedback will help improve AI systems. Prior AI... ...tools is an advantage. # Strong analytical skills and the ability to provide clear...SuggestedRemote jobFor contractors
$70 - $110 per hour
...technical talent with leading AI research labs. Headquartered... ...Role Responsibilities Evaluate AI-generated financial plans... ..., regulatory compliance, and analytical rigor. Review financial scenarios... ...reporting. Provide expert feedback to improve AI model...SuggestedHourly payContract workSummer workWork at officeImmediate startRemote work$30 per hour
...elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ..., answering questions, and correcting errors. Evaluate the experience and deliverables, providing concise feedback on performance. Qualifications Must-Have...SuggestedRemote jobHourly payWeekly payContract workFor contractorsSummer work$40 - $80 per hour
...Description Job Description Job Description: Paralegal – AI Training & Legal Evaluation Role Type: Contractor Location: Remote (Worldwide,... .... # Familiarity with AI or LLM evaluation. # Strong analytical, organizational, and problem-solving skills....SuggestedRemote jobHourly payFor contractorsWorldwide10 hours per weekFlexible hours$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Location: Remote Role Responsibilities Evaluate the quality and correctness of AI-... .... Provide clear, rubric-based written feedback to improve AI model outputs . Review...SuggestedRemote jobContract workSummer work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...regulatory response with financial-services AI Evaluator Type: Contract Compensation: $... .... Provide clear, structured written feedback to improve AI model outputs. Apply deep...Contract workSummer workWork at officeRemote work$137.9k - $256.1k
Sr. Legal AI Prompter / EvaluatorJob DescriptionSr. Legal AI Prompters / Evaluators are responsible for improving CoCounsel Legal by writing and refining prompts, skills... ...retrieval, or content), and provide strategic feedback to software engineers and data...Full timeWork at officeLocal areaFlexible hours$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $... ...outputs. Provide clear, structured written feedback to improve quality. Work independently...Contract workSummer workWork at officeRemote work$70 - $110 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...+ hours/week Role Responsibilities Evaluate AI-generated operational plans , staff... ...and workflow coordination. Provide expert feedback. Leverage real-world expertise in directing...Hourly payContract workSummer workImmediate startRemote work$60 - $70 per hour
...technical talent with leading AI research labs. Headquartered... ...Role Responsibilities Evaluate AI-generated responses for... ...violations. Provide structured feedback to improve model alignment and... ..., critical thinking, and analytical reasoning skills. ~ Ability...Contract workSummer workRemote work$20 - $160 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $... ...Preferred Experience in annotation, human-feedback, evaluation, or content review....Contract workSummer workRemote work$80 - $120 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...and Jack Dorsey . Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour...Contract workSummer workWork at officeRemote work$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Biology / environmental science Evaluator Type: Contract Compensation: $... ...outputs. Provide clear, structured written feedback to improve AI-generated content....Contract workSummer workWork at officeRemote work- YO AI Labs is seeking experienced Humanitarian Aid Representatives to contribute expertise in humanitarian operations, civilian protection... ...and International Humanitarian Law (IHL) to an AI training and evaluation project. You will help evaluate and refine AI-generated...Remote job
$70 per hour
...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers...Contract workSummer workImmediate startRemote work$160 per hour
...and technical talent with leading AI research labs. Headquartered in... ...relevance. Provide structured feedback to improve document standards and evaluation processes. Utilize Excel ,... ...in a relevant domain ~ Strong analytical and written communication skills...Remote jobContract workSummer work$190 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Remote Commitment: 20+ hours/week Role Responsibilities Evaluate AI systems on complex personal workflows, including personal...Remote jobHourly payContract workFor contractorsSummer workTrial period$50 per hour
...Prolific Academic Ltd is seeking Product Designers and UX Specialists to join our Expert Network to train and evaluate AI models using real design expertise. If selected, you’ll complete a brief test and, upon success, join Prolific as a paid participant to work on AI...Hourly payRemote workWork from homeFlexible hours- ...Description Job Title: Language Model Evaluator Job Summary We are seeking a... ...Models (LLMs) and Generative AI applications. The ideal... ..., and test cases. Strong analytical skills with the ability to... ...reinforcement learning from human feedback (RLHF) concepts....Full timeRemote work
$100 - $120 per unit
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...and Jack Dorsey . Position: Baseball Fan – Live MLB Game AI Evaluator (US) Type: Contract Compensation: $100–$120 per task...Contract workWork experience placementSummer workRemote work$80 - $150 per hour
...technical talent with leading AI research labs. Headquartered... ...Position: MS Excel / Google Sheets Evaluator Type: Contract... ...relevance. Provide structured feedback to improve document... ...and Google Slides ~ Strong analytical and written communication skills...Contract workSummer workRemote work$50 per hour
...technical talent with leading AI research labs. Headquartered... ...Role Responsibilities Evaluate the quality and accuracy of... ...and language gaps, providing feedback to improve LLM performance... ...Strong attention to detail, analytical judgment, and written communication...Contract workSummer workRemote work- ...Job Description Job Description Role Title: AI Agent Evaluator Role Type: Contractor Location: United States We are seeking detail... ...agents side by side, assess their performance, and provide feedback to improve their reliability and user experience. Key...Remote jobFor contractorsWork at office
- YO AI Labs is seeking experienced physicians to support an advanced healthcare AI project. This part-time contractor role involves applying clinical expertise to evaluate AI-generated medical content, analyze cases, and improve AI systems used in healthcare decision-making...Remote jobPart timeFor contractors
$60k - $100k
...mindset—eager to learn, adapt, and evolve with the company Ability to work independently while remaining open to feedback and process improvements Strong analytical skills and attention to detail Proficiency with appraisal software and market research tools A professional...For contractors$12 per hour
Turkish AI Product Evaluator Productive Playhouse is building a talent pool of Turkish speakers for an upcoming project testing and evaluating leading AI chatbots. The objective is to enhance response quality through direct user interaction with the various AI models....Hourly payFor contractorsFreelanceRemote workWorldwideFlexible hours- ...Web Browsing Evaluator Contractor Remote Micro1 is engaging Web Browsing Evaluators... ...expertise to help train next-generation AI systems. Your work will shape how models... ...Provide clear written documentation and feedback on the browsing process, highlighting any...For contractorsRemote work
- ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Commitment: Flexible part-time hours Role Responsibilities Evaluate AI answers to endocrinology questions for clinical accuracy,...Contract workPart timeSummer workImmediate startRemote workFlexible hours
- Luxury Brand Evaluator Turn your passion for luxury into a career opportunity! Explore the world of premium brands and make a lasting... ...to discreetly assess customer experiences, providing critical feedback that helps brands refine their services. Whether visiting boutiques...Worldwide
$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review... ...ETL pipelines , data warehouses , analytics platforms , and distributed data systems...Contract workSummer workRemote work
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!



