Analytical Evaluator - AI Feedback [Remote]
$100 per hourMercor
New York, NY
- Remote job
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Generalist Expert
Type: Contract
Compensation: $100/hour
Location: Remote
Role Responsibilities
- Evaluate AI-generated responses to enhance reasoning and rigor in model outputs .
- Provide structured written feedback to improve training data quality and downstream performance.
- Develop clear, precise, and well-evidenced written rationales beyond surface-level observations.
- Collaborate with AI research teams to ensure consistency and coverage across datasets.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- Bachelor's degree from a top-100 globally ranked university preferred.
- Strong analytical and written communication skills.
- Ability to work independently and follow detailed task guidelines.
- Strong critical reading skills with the ability to identify nuance, implicit meaning, and gaps in reasoning.
- Ability to write clear, precise, and well-evidenced written rationales.
- Consistent and honest judgment, including the ability to give critical assessments.
- Strict attention to detail and accurate application of structured evaluation guidelines.
- Ability to work entirely without AI writing tools .
- Native English fluency required.
Preferred
- Candidates based in the United States , United Kingdom , Canada , Australia , or New Zealand .
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: Show email
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 19 days ago
Similar jobs that could be interesting for youBased on the Analytical Evaluator - AI Feedback [Remote] in New York, NY vacancy
$60 per hour
...A leading AI development firm is seeking experienced quantitative... ...team. The role involves evaluating AI-generated quantitative work... ...for AI training, and providing feedback on model performance.... ...coding experience, and strong analytical skills. This position allows...SuggestedHourly payRemote workFlexible hoursDataAnnotation
Brooklyn, NY5 days ago- ...for a role focused on assessing AI-generated responses. Responsibilities include evaluating reasoning quality and factual... ...as well as providing structured feedback to improve AI models. The position... ...for candidates with strong analytical skills and attention to detail....SuggestedFull timePart timeFlexible hours
Crossing Hurdles
New York, NY2 days ago $100 per hour
...for a remote contract position. The role involves evaluating AI-generated responses, providing structured feedback to enhance training data quality, and... ...Bachelor's degree from a top-100 university, strong analytical and communication skills, and the ability to work...SuggestedRemote jobContract work- ...Mercor is seeking an analytical thinker in Kentucky to evaluate AI-generated responses and provide structured feedback. Candidates will contribute to high-impact AI research projects and should possess a strong analytical mindset and excellent communication skills....Suggested
Mercor Inc
Brooklyn, NY3 days ago $14.5 per hour
...Join to apply for the AI Web Search Evaluator role at Welo Data Welo Data works with technology... ...of attention to detail, strong analytical skills, and a passion for quality.... ...they meet user expectations. Offer feedback and recommendations to improve algorithm...SuggestedHourly payPart timeImmediate startRemote workWork from home10 hours per weekFlexible hoursWelo Data
New York, NY4 days ago$40 per hour
...professionals to join our team to help train AI models. In this role, you will evaluate AI-generated security content,... ...problems, and provide feedback to improve how AI systems reason about... ...bilingual level) Strong writing and analytical skills A bachelor’s degree is...Hourly payFull timePart timeRemote workDataAnnotation
Brooklyn, NY5 days ago$14.5 per hour
...A technology company is seeking an AI Web Search Evaluator to enhance the quality of search engine results.... ...analyzing search performance and providing feedback to improve algorithms. Ideal candidates will have strong analytical skills and familiarity with online search...Hourly payPart timeRemote workFlexible hoursWelo Data
New York, NY4 days ago- ...A leading AI development firm is seeking experienced quantitative professionals to evaluate AI-generated analyses and contribute to training robust AI models. This remote role offers flexibility and competitive pay, with opportunities for those from diverse quantitative...Remote work
DataAnnotation
New York, NY4 days ago $60 per hour
...contribute to developing cutting-edge AI systems, while enjoying the... ...of performing complex analytical and scientific reasoning — but... ...-art AI models on tasks like evaluating AI-generated quantitative... ...technical problems, and providing feedback that directly shapes how...Hourly payFull timeRemote workFlexible hoursDataAnnotation
Brooklyn, NY5 days ago- A leading AI research firm in New York seeks a part-time Legal Evaluator. In this role, you will evaluate the quality and relevance... ...while providing structured feedback. Candidates should have... ...law or legal research, strong analytical skills, and excellent written communication...Part time
- ...Italian Data Trainer to enhance AI systems' safety and reasoning... .... Responsibilities include evaluating AI-generated content,... ...evaluations, and providing expert feedback. Candidates should possess a... ...experience in Trust & Safety and analytical skills, and offers a flexible...Remote jobFlexible hours
- ...hourly-paid contractor role. The candidate will evaluate AI-generated content, providing structured feedback to enhance safety, reliability, and reasoning quality... .... Experience in Trust & Safety and strong analytical skills are essential. #J-18808-Ljbffr YO IT ConsultingRemote jobHourly payFor contractors
$80 - $100 per hour
A leading AI research matchmaking firm is seeking a part-time Legal Evaluator in New York. The role involves evaluating the quality... ...reports, providing structured feedback, and ensuring clarity and... ...law or legal research, strong analytical capabilities, and excellent written...Hourly payPart timeFlexible hours- ...IT Consulting is looking for a Visual Evaluation Specialist to evaluate and annotate visual content, assist in AI training, and provide feedback for improvements. The ideal candidate... ...15 hours a week. You'll leverage your analytical skills to assess visual materials while...Remote jobFor contractors
- ...IT Consulting is seeking a Generalist Expert to evaluate AI-generated responses and provide structured feedback. Candidates will have the opportunity to contribute... .... The ideal candidate will possess strong analytical skills, native English fluency, and the ability...Remote jobFor contractors
- ...TransPerfect is looking for game enthusiasts to help train AI by reviewing its sports-related outputs. This remote, freelance position involves evaluating AI performance based on factual accuracy and clarity, shaping how AI interacts in sports discussions worldwide. Ideal...FreelanceRemote workWorldwide
TransPerfect
New York, NY4 days ago $40 per hour
...remote role focused on training AI models related to security. Responsibilities include evaluating AI-generated content, solving... ..., and providing constructive feedback. Candidates should have over 2... ...cybersecurity, coding skills, and strong analytical capabilities. This position...Hourly payRemote workFlexible hoursDataAnnotation
Brooklyn, NY5 days ago$20 per hour
A tech company specializing in AI evaluation is looking for an Interface Designer to enhance AI systems' understanding of design. This role involves reviewing AI-generated designs and providing feedback to improve aesthetic quality. The position is flexible, allowing you...Remote workFlexible hoursDataAnnotation
New York, NY4 days ago$40 per hour
...allows flexibility to work on various projects while evaluating AI-generated content and offering feedback to improve cybersecurity systems. The ideal... ...years in cybersecurity, coding experience, and strong analytical skills. Compensation starts at $40+ per hour, and applicants...Hourly payRemote workDataAnnotation
New York, NY5 days ago$11.5 per hour
...position as an Online Task Contributor. In this role, you will evaluate and provide feedback on content to enhance search engine results and quality. No... ..., based on task completion, with a supportive community of contributors involved in AI advancements. #J-18808-Ljbffr...Hourly payPart timeRemote workUniversity of Delaware
New York, NY2 days ago$40 per hour
...company is seeking experienced cybersecurity professionals to evaluate AI-generated security content and solve technical problems. The... ...have 2+ years of hands-on experience in cybersecurity, strong analytical skills, and preferred coding experience. Paid projects start...Hourly payFull timePart timeRemote workFlexible hoursDataAnnotation
New York, NY5 days ago$60 per hour
...cybersecurity professionals to join their remote team. In this role, you will evaluate AI-generated cybersecurity content, design and solve technical problems for AI systems, and provide feedback that influences AI model development. Candidates should have a minimum of 2...Hourly payRemote workFlexible hoursDataAnnotation
New York, NY2 days ago- ...is seeking experienced Legal professionals to contribute to an AI project aimed at enhancing AI's understanding of complex legal concepts... .... Candidates must have an advanced legal education and strong analytical skills. Compensation is $110–$130/hour, with potential bonuses...Part timeRemote work
Call For Referral
New York, NY2 days ago - ...building a global network of domain experts to support high-quality AI evaluation across training, benchmarking, red‑teaming, and ongoing model... ...job descriptions English fluency is required for guidelines, feedback, and collaboration AI is changing how the world communicates...Contract workLocal areaFlexible hours
LILT (Production)
New York, NY2 days ago $40 per hour
...A leading AI annotation company is seeking experienced quantitative professionals to evaluate AI-generated quantitative work and contribute to the development of cutting... ...+ years of quantitative experience, strong analytical and coding skills, and a background in areas...Remote workFlexible hoursDataAnnotation
New York, NY4 days ago$40 per hour
...leading cybersecurity firm is seeking experienced professionals to evaluate AI-generated security content and solve technical problems in... ...2 years of experience in cybersecurity, strong writing and analytical skills, with a preference for a bachelor's degree and...Hourly payRemote workDataAnnotation
New York, NY4 days ago- ...-focused company is looking for experienced professionals to evaluate AI-generated security content and contribute to the development... ...cybersecurity roles and must possess strong coding, writing, and analytical skills. This position offers the flexibility of remote work...Hourly payRemote work
DataAnnotation
New York, NY4 days ago $50 - $60 per hour
...DataAnnotation is committed to creating high-quality AI. We are looking for a Sales & Trading... ...it doesn’t. Share clear, structured feedback to help make each new version of the AI... ...diverse and complex problems and evaluate their outputs Evaluate the quality produced...Hourly payFull timeContract workPart timeWork experience placementRemote workFlexible hoursDataAnnotation
Brooklyn, NY5 days ago$50 - $60 per hour
...States is seeking a Chief Information Officer (CIO) to enhance AI capabilities, especially in finance. This role is ideal for experienced... ...professionals who can improve AI Assistant models through feedback on macro trends and capital markets. Enjoy the flexibility of remote...Hourly payRemote workFlexible hoursDataAnnotation
Brooklyn, NY5 days ago- ...experienced cybersecurity professionals for a remote role, focusing on evaluating AI-generated security content and solving technical problems in... ...penetration testing or threat analysis, strong writing and analytical skills, and a bachelor's degree is preferred. This position...Hourly payRemote work
DataAnnotation
Brooklyn, NY5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Analytical Evaluator - AI Feedback [Remote]. Be the first to apply!
Related searches
- quality evaluator New York, NY
- evaluator New York, NY
- speech language pathologist evaluator New York, NY
- program evaluator New York, NY
- clinical evaluator New York, NY
- work from home web search evaluator New York, NY
- work from home social media evaluator New York, NY
- social media evaluator New York, NY
- remote coding part time New York, NY
- franchise development manager (remote) New York, NY