Analytical Evaluator - AI Feedback
$70 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $70/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to strengthen reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales that go beyond surface-level observations.
- Ensure consistent and honest judgment, including critical assessments when warranted.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-500 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Native English fluency required.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback in San Francisco, CA vacancy
$80 - $120 per hour
Mercor is seeking a User/Customer Research and Feedback Synthesis Evaluator to evaluate AI-generated artifacts using specific quality rubrics. The role demands deep subject-matter expertise in user research and involves providing feedback to improve AI performance. The...SuggestedRemote jobContract workWork at officeFlexible hours$60 - $70 per hour
...technical talent with leading AI research labs. Headquartered... ...Role Responsibilities Evaluate AI-generated responses for... ...violations. Provide structured feedback to improve model alignment and... ..., critical thinking, and analytical reasoning skills. ~ Ability...SuggestedContract workSummer workRemote workMercor
San Francisco, CA6 days ago- We are seeking experienced AI Safety Practitioners to evaluate the safety, quality, and alignment of frontier... ...through structured evaluations and feedback. Responsibilities Evaluate AI-... ...written English, critical thinking, and analytical reasoning skills. Ability to...SuggestedWorldwide
- Obsidian is hiring expert Evaluators in Investment analysis / valuation / credit to review AI-generated work products for accuracy and quality. This remote, hourly... ...professional fluency in English to provide structured feedback. The ideal candidate will have over 5 years of...SuggestedHourly payWork at officeRemote work
- Obsidian is looking for expert Evaluators in Finance operations/audit support to review AI-generated work products for accuracy and quality. This remote hourly... ...involve evaluating outputs and providing structured feedback. Preferred candidates will hold advanced degrees...SuggestedRemote jobHourly payWork at office
- Obsidian seeks experienced AI Safety Practitioners to evaluate safety, quality, and alignment of frontier AI models across grey-area topics. You will... ...model behavior through structured evaluations and feedback. Responsibilities include evaluating safety, factual accuracy...
$70 - $110 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...+ hours/week Role Responsibilities Evaluate AI-generated operational plans , staff... ...performance improvement decisions. Provide expert feedback. Work independently and...Hourly payContract workSummer workImmediate startRemote workMercor
San Francisco, CA2 days ago- Obsidian is hiring expert Evaluators in Real estate, hospitality, and events to review AI-generated work for accuracy, rigor, and domain quality. This remote position... ...Workspace. This role emphasizes quality evaluation and structured feedback. #J-18808-Ljbffr ObsidianRemote jobWork at office
$80 - $120 per hour
Mercor is seeking a Media / journalism / communications Evaluator to evaluate AI-generated artifacts and provide structured feedback. Ideal candidates will have over 5 years of experience and must be fluent in English. This remote role requires proficiency in Microsoft...Remote jobHourly payWork at office- Obsidian is seeking expert Evaluators in FP&A / corporate finance to assess AI-generated work products for accuracy and quality. This role entails deep expertise to grade outputs and provide structured feedback. Candidates should have at least 5 years of relevant experience...Remote jobHourly payWork at office
- Obsidian is seeking expert Evaluators in Biology/environmental science to review and assess AI-generated work products for accuracy and quality. In this remote, hourly... ..., you will leverage your expertise to provide feedback on documents and presentations, ensuring they...Remote jobHourly pay
$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $... ...errors. Provide clear, structured written feedback. Collaborate with subject matter experts...Contract workSummer workWork at officeRemote workMercor
San Francisco, CA14 days ago$80 - $120 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $... ...decks. Provide clear, structured written feedback to improve AI-generated work products....Contract workSummer workWork at officeRemote workMercor
San Francisco, CA3 days ago- Mercor is seeking experienced musicians to evaluate generative music AI models in partnership with a leading AI lab. You will assess AI-generated lyrics across a wide range of genres and rate them against detailed quality standards, working in Malayalam and English. Required...
- Synthires is offering a part-time role for PhD-level Chemistry experts to contribute to AI safety and evaluation projects. The work involves applying scientific expertise to understand and improve how AI systems handle specialized chemistry topics, with training provided...Remote jobPart time
- Mercor is seeking experienced AI Safety Practitioners to assess the safety, quality, and alignment... ...complex, policy-sensitive topics. You will evaluate AI-generated responses, apply safety policies, and provide structured feedback to improve model behavior. Responsibilities...
- Welo Data is seeking Data Labeling Associates in California to evaluate AI outputs and ensure cultural context and safety in Arabic datasets. This role requires professional-level proficiency in Portuguese (Brazil), a bachelor's degree, and at least 2 years of experience...
- Obsidian is seeking a Spanish Audio Generalist Evaluator Expert to contribute to a high-impact audio AI research project. You will handle transcription, annotation, and evaluation tasks to help train and benchmark advanced language models. The ideal candidate should have...Part time10 hours per week
$80 - $120 per hour
Mercor is looking for a Biology / environmental science Evaluator to assess AI-generated artifacts based on quality rubrics. This position requires evaluating documents for errors and collaborating with AI teams to improve model performance. The ideal candidate should...Remote jobHourly payContract workWork at office$70 per hour
...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers...Contract workSummer workImmediate startRemote workMercor
San Francisco, CA2 days ago- Welo Data in San Francisco is hiring Data Labeling Associates for Project Perseus. This role focuses on evaluating Arabic AI systems, requiring professional proficiency in Portuguese and experience in AI safety. Responsibilities include assessing AI outputs, identifying...Full time
- ...for pre‑sales deliverables and to score AI‑generated and human work samples with detailed... ...plans, demos, proofs of concept, and evaluation plans. The role emphasizes clear written... ...the ability to iterate quickly based on feedback from senior reviewers. #J-18808-Ljbffr Obsidian
$50 - $75 per hour
A leading tech company based in Australia is seeking an AI Model Evaluator on a contract basis. The role involves evaluating AI-generated responses, writing prompts, and providing justifications based on specific criteria. Ideal candidates will hold a Master's degree in...Hourly payContract work- A leading AI research accelerator is seeking an experienced medical professional to leverage expertise in internal or emergency medicine... ...AI model development. You will design clinical scenarios and evaluate AI-generated responses, ensuring impactful AI solutions in...Remote jobContract work
- Mercor is hiring experienced music professionals to evaluate generative music AI models, in partnership with a leading AI lab. You will assess AI-generated music across a wide range of genres and rate it against detailed quality standards, working in Thai and English....Flexible hours
- Obsidian is collaborating with a leading AI research group to engage experts with advanced training in physics. The role involves solving complex physics problems and reviewing AI-generated proofs. Ideal candidates should hold a PhD in Physics from a top program and possess...
- Mercor seeks experienced music producers and audio engineers to evaluate generative music AI models in collaboration with a leading AI lab. You'll assess AI-generated music across diverse genres, label musical characteristics, review lyrics and vocals, and rate quality...Immediate startFlexible hours
- Mercor is hiring experienced musicians to evaluate generative musical AI models in partnership with a leading AI lab. You will assess model outputs across lyrics, voice generation, and other standards, using your bilingual language skills. Ideal candidates have 3+ years...Part timeImmediate start10 hours per week
- Mercor is seeking experienced Clinical Law Professors and Clinic Directors to evaluate AI-generated legal reasoning in civil legal services. Experts will blend doctrinal knowledge with practical supervision to assess AI analyses. Applicants should hold a JD with active...Remote job
- Mercor is seeking experienced musicians to evaluate generative musical AI models in collaboration with a leading AI lab. You will assess model outputs across different categories of music in your bilingual language and contribute to structured taxonomy annotations. Ideal...Part timeImmediate start10 hours per week
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback. Be the first to apply!
Related searches
- work from home web search evaluator San Francisco, CA
- social media evaluator San Francisco, CA
- evaluator San Francisco, CA
- quality evaluator San Francisco, CA
- education evaluator San Francisco, CA
- ai evaluator San Francisco, CA
- transcript evaluator
- work from home social media evaluator
- work from home web search evaluator
- social media evaluator