Remote AI Evaluation Specialist - Critical Reasoning
Mercor Inc
New York, NY
- Remote job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, this remote contract position seeks a Generalist Expert to evaluate AI-generated responses, strengthening reasoning and rigor in model outputs. Responsibilities include providing structured written feedback, improving training data quality and downstream performance, and developing clear, well-evidenced rationales. #J-18808-Ljbffr Mercor Inc
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Remote AI Evaluation Specialist - Critical Reasoning in New York, NY vacancy
- ...looking for a Personal Injury Law & AI Evaluation Specialist based in the United States. In this remote contract role, you will... ...conclusions, and weaknesses in legal reasoning. Your structured feedback will... ...legal professional who enjoys critical analysis, precision, and the...Remote workContract workFor contractorsFlexible hours
- AI Evaluation Specialist Role Type: Contractor Location: Remote — US, Canada, UK, Ireland, Australia, New Zealand We are seeking... ...volumes of examples. Identify reasoning gaps, tool-use issues, logical... ...or quality assessment. Strong critical-thinking skills with a focus on...Remote jobFor contractors
- ...experienced Employment Law Specialist to contribute to a... ...the legal reasoning, accuracy, and... ...reliability of advanced AI systems. In this... ...law scenarios, evaluate AI-generated legal... ...English LOCATION: Remote — United States... ...issue-spotting and critical analysis skills when...Remote workContract workFor contractors
- ...looking for an Employment Law & AI Evaluation Specialist based in the United States. In this remote contract role, you will apply... ...accuracy, reliability, and reasoning capabilities of advanced artificial... ...legal reasoning. Apply critical legal analysis: Assess...Remote workContract workFor contractorsFlexible hours
- ...partner is looking for a Tax Law & AI Evaluation Specialist based in the United States. In this remote contract role, you will... ...the accuracy, reliability, and reasoning capabilities of advanced artificial... ...Analytical skills: Strong critical-thinking capabilities and the...Remote workContract workFlexible hours
- ...Ophthalmology AI Evaluation Specialist — Clinical Reasoning Review is a remote evaluation track for reviewing ophthalmology ai evaluation evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the...Remote jobHourly payFor contractors10 hours per week
- ...Location: India/Pakistan (Remote) Compensation: $5.00... ...Shape the Future of AI in Finance Volga... ...development, training, and evaluation of next-generation financial and insurance reasoning models. If you have... ...Strong analytical and critical-thinking skills. Excellent...Remote workHourly payExtra incomePermanent employmentContract workTemporary workFlexible hours
Volga Partners
Remote5 days ago $70 per hour
...technical talent with leading AI research labs.... ...Compensation: $70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses to... ...ensure accuracy and depth in reasoning. Provide structured... ...detailed task guidelines. Strong critical reading skills. Native...Remote workContract workSummer work$20 - $80 per hour
...Help improve next-generation AI systems by supplying precise, real-world evaluation, annotation, and feedback. This remote contractor role focuses on how AI models learn, reason, and perform across diverse... ...and evaluations. Critically read and analyze complex material...Remote workHourly payContract workFor contractors- ...world’s fastest-growing AI companies accelerating... ...capabilities in thinking, reasoning, coding, agentic... ...systems that solve mission-critical priorities for companies... ...project involving the evaluation of multiple chatbot responses... ...and finance. Fully remote and flexible work...Remote workFreelanceFlexible hoursWeekday work
Braintrust
Austin, TX1 day ago - A leading AI research accelerator is seeking Geospatial Experts... ...to enhance AI systems through evaluations and real-world applications. This... ...contractor position is fully remote with flexible hours, primarily focusing on geospatial reasoning tasks. Responsibilities...Remote jobFor contractorsFlexible hours
- We are hiring: Remote Dermatology AI Evaluation Specialist — 24-Mag Llc — Remote. Type: Full-Time. This is an excellent opportunity for the right candidate... ..., safety, and clinical relevance, review diagnostic reasoning, and provide structured feedback. This freelance...Remote workFull timeFreelance
24-Mag Llc
Remote2 days ago $30 per hour
BANGLA AI EVALUATION SPECIALIST Up to $30/hr Depending on the project Bachelor's Remote Part-time Flexible hours Opportunity Overview Handshake is looking for Bangla speakers... ..., sharing your honest preferences and reasoning Provide clear written feedback that helps AI...Remote workPart timeLocal areaFlexible hours$34 per hour
...be working directly with cutting-edge AI systems — evaluating outputs, identifying gaps, and helping... ...judgment , where your ability to think critically matters just as much as following... ...Important : This is a 100% onsite position — remote work is not available for this role....Remote workFull timeContract workVisa sponsorship- ...k+ experts train the world’s leading AI models. You’ll evaluate AI on Slovak translation tasks, review... ...the next generation of AI. Flexible, remote, and always project-available. You’ll... ...errors, explain failures, and score reasoning. Write clearer, correct answers when...Remote jobFlexible hours
$60 per hour
...part-time QA project focused on autonomous AI agents. This flexible remote opportunity requires strong analytical and critical thinking skills, alongside attention to detail... ...data formats. Candidates will review evaluation tasks, identify inconsistencies, and help define...Remote jobPart timeFlexible hours- ...next steps. Our partner is looking for an AI Evaluation Engineer based in United States. As a... ...automated testing systems for complex reasoning, multi-turn conversations, and agent interactions... ...with modern AI technologies in a fully remote, globally distributed environment. Your...Remote workHourly payFor contractorsFlexible hours
- e2f Inc. is seeking an AI Research & Evaluation Analyst for remote, project-based work. You will support research and rubric development for advanced AI... ...model projects, requiring strong English writing, careful reasoning, and reliable sourcing. Responsibilities include...Remote job
- OpenTrain AI, Inc. is seeking a remote contractor to evaluate AI-generated SQL queries, database designs, and data-processing logic, focusing on correctness... ...will create strong reference answers and explain your reasoning, review methodology and indexing, and provide step-...Remote jobFor contractors
- ...Benchmark & Evaluation Engineer Type: Full-time Location: Remote (Philippines) Schedule: US Shift We are a tech-enabled... .... Our client is an innovative AI platform company developing... ...PII to test query scalability and reasoning. Curate Task Libraries: Write...Remote workFull timeShift work
Expedock
United States5 days ago $60 - $70 per hour
...technical talent with leading AI research labs.... ...$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses... ...outputs, hallucinations, reasoning failures, and policy violations... ...Excellent written English , critical thinking, and analytical...Remote workContract workSummer work- ...Job Title AI Evaluation Engineer Location Hybrid / Remote Employment Type Full-time Job Summary We are seeking... ...) Soft Skills Analytical thinking Critical thinking Problem-solving Strong communication...Remote workFull time
Ova Technologies
New York, NY5 days ago $35 - $42 per hour
...Location: USA (Remote) Compensation: $35.00–$42... ...Shape the Future of AI in Finance, Audit & Risk... ...network of on-call AI Evaluation Specialists supporting the... ...generated responses, apply critical thinking, and provide... ...for factual accuracy, reasoning quality, and business...Remote workHourly payExtra incomePermanent employmentContract workTemporary workFlexible hoursVolga Partners
Remote29 days ago$20 per hour
...Job Description Job Description We are hiring remote AI Model Evaluators and AI Trainers to support multiple ongoing AI training projects.... ...browser-based environments Evaluate AI outputs for accuracy, reasoning, clarity, tone, and instruction following Follow...Remote workPart timeFor contractorsWork experience placementFlexible hoursDraft labs
Miami, FL10 days ago- ...expertise to support the training and evaluation of next-generation AI systems. You will analyze tax law... ...documents, identify errors or gaps in legal reasoning, and provide structured feedback to... ...written English LOCATION: Remote — United States only Key Responsibilities...Remote workContract work
$70 - $100 per hour
...technical talent with leading AI research labs. Headquartered in... ...$70–$100/hour Location: Remote Commitment: 10–20+ hours/... ...select the stronger one. Evaluate both the content and rendering... ...where document quality is mission-critical. Familiarity with comparing...Remote workContract workSummer workWork at officeImmediate startMercor
Dallas, TX9 days ago$30 per hour
...Prolific is seeking Fluent Korean Speakers to join our Expert Network. You will help train and evaluate AI models by completing paid tasks and passing a quick skills test to join as a participant. You can work from home and earn competitive pay. Researchers paying for...Remote workSelf employmentWork from homeProlific
New York, NY4 days ago$152k - $241.5k
...are foundational to American AI leadership and cybersecurity,... ...Security Engineering team builds and evaluates AI-powered tooling that helps... ...the program. You will play a critical role in separating real... ...US, CA, Santa Clara; US, NC, Remote; US, NY, Remote; US, TN, Remote...Remote workFull timeNvidia
Raleigh, NC4 days ago- AI Trainer Jobs seeks a Generalist (Macbook User) to remotely evaluate prompts and responses against AuraOne's rubric. You will compare paired outputs, label edge cases, and draft structured feedback for retraining. Role involves reviewing frontier model outputs, judging...Remote job
$400k
...researchers to design rigorous evaluation frameworks for trustworthy healthcare-focused AI systems. You will develop benchmark... ..., and analyse medical reasoning to improve AI reliability within clinical workflows. This fully remote, full-time engagement offers compensation...Remote jobFull time
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Remote AI Evaluation Specialist - Critical Reasoning. Be the first to apply!
Related searches
- ai data scientist New York, NY
- ai scientist New York, NY
- software engineering manager remote New York, NY
- remote recruitment consultant New York, NY
- remote data manager New York, NY
- business intelligence analyst remote New York, NY
- benefits specialist remote New York, NY
- junior angular developer remote New York, NY
- remote video game New York, NY
- remote typing work New York, NY