Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Data Science Expert - Evaluation Specialist

$120 - $170 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Data Science Expert
Type: Contract
Compensation: $120–$170/hour
Location: Remote

Role Responsibilities

  • Design precise, task-specific grading criteria for real-world data science deliverables, including analyses, models, dashboards, and experiment readouts.
  • Score AI-generated and human work samples against established criteria with detailed, well-reasoned written justifications.
  • Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
  • Incorporate structured feedback from senior reviewers and iterate quickly on your work.
  • Collaborate with AI research teams to improve training data quality and downstream performance.

Qualifications

Must-Have

  • 5+ years of professional data science experience in industry.
  • Background in business operations, product, or growth data science at top-tier technology companies.
  • Deep fluency in experiment design, A/B testing , metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
  • Exceptionally strong written communication.
  • Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.

Preferred

  • Prior experience with AI training , evaluation, or human-data projects.

Application Process (Takes 20–30 mins to complete)

  • Submit your resume or relevant technical background to get started.
  • Qualified applicants may be asked to complete a brief technical assessment or submit additional information.

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 10 days ago
Similar jobs that could be interesting for youBased on the Data Science Expert - Evaluation Specialist in New York, NY vacancy
  • $120 - $170 per hour

     ...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $120–$170/hour...  .... Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    2 days ago
  • $45 - $55 per hour

     ...the project channel. Qualifications Must-Have Able to commit 30 hours per week . Prior experience with AI data work . Sharp eye for detail and patience to double-check work. Track record in detail-heavy work such as operations accounting... 
    Suggested
    Full time
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    25 days ago
  • Crewlia is seeking an AI Data Analyst for a remote hourly contract in the United States and Canada. You will assess AI-generated data...  ...reasoning, with flexible hours and clear project guidelines. You will evaluate outputs against source materials, documenting rationales and any... 
    Suggested
    Remote job
    Hourly pay
    Contract work
    Flexible hours

    Crewlia

    New York, NY
    1 day ago
  • Seoulstart is seeking a Data Analyst (AI Evaluation) to join our remote team in the United States. You will review datasets and outputs for quality, apply evaluation rubrics, and provide clear feedback to improve AI training data. The role emphasizes attention to detail... 
    Suggested
    Remote job
    Hourly pay

    Seoulstart

    New York, NY
    4 days ago
  • $25 - $50 per hour

     ...collaboration About micro1 micro1 is a data lab that helps AI labs &...  ...agents. We provide frontier evaluations and reinforcement learning...  ...sources and vets domain experts for our data platform that enables...  ...Engineering & physics Earth sciences & GIS Data science & statistics... 
    Suggested
    For contractors
    Remote work
    Flexible hours

    Micro1

    New York, NY
    4 days ago
  • $50 - $80 per hour

    The work You will help evaluate how well AI handles real-world business analysis tasks. The work combines data privacy review, analytical quality checks, and clear scoring standards for AI performance. Review business datasets for personally identifiable information and... 
    Hourly pay
    Part time
    For contractors
    Remote work

    OpenTrain AI, Inc.

    New York, NY
    4 days ago
  • Turing is seeking a Business Analyst to review AI responses and validate information accuracy for an evaluation project. You will assess AI outputs, perform online research, and provide structured annotations and feedback to improve model performance in reasoning and task... 
    Hourly pay

    Turing

    New York, NY
    8 hours ago
  • Crossing Hurdles is seeking a Business Analyst for a remote, short-term contract. The role involves evaluating chatbot outputs against criteria, conducting structured analyses, and delivering clear assessments to enhance AI response quality. You will collaborate with cross... 
    Remote job
    Temporary work

    Crossing Hurdles

    New York, NY
    4 days ago
  • $45 - $55 per hour

     ...Location: Remote Role Responsibilities Evaluate the accuracy and depth of AI-generated...  ...AI research teams to improve training data quality and downstream performance....  ...solutions. Collaborate with subject matter experts to ensure consistency, relevance, and... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    5 days ago
  • $20 - $50 per hour

    At TELUS Digital, data is the lifeblood of our innovation...  ...Architect and execute evaluation frameworks to measure...  ...’s Degree in Data Science, Mathematics,...  ...Currently, pay rates for experts range from $20 - $50 USD...  ...Collaborate with a network of specialists across 20+ domains and... 
    Hourly pay
    For contractors
    Remote work
    Flexible hours

    Telus Digital AI Solution

    New York, NY
    18 hours ago
  • $200k - $300k

     ...River Trading (HRT) is seeking a Research Data Expert to join our Data team. Data is a...  ...with the sourcing team to identify and evaluate new data procurement opportunitiesAnalyze...  ...Python/R preferred) and proficiency in data science methodologies5+ years of professional experience... 
    Work at office
    Local area
    Immediate start

    Hudson River Trading

    New York, NY
    3 days ago
  •  ...About the role Calibration Technology Evaluation Specialist is a remote review track for evaluating AI outputs across calibration technology evaluation specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational... 
    Hourly pay
    For contractors
    Work experience placement
    Remote work
    10 hours per week

    AuraOne

    New York, NY
    1 day ago
  • $75 - $110 per hour

     ...Neurology Evaluation Specialist is a remote clinical-review track for evaluating AI outputs that touch neurology. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning so... 
    Hourly pay
    For contractors
    Remote work
    10 hours per week

    AuraOne

    New York, NY
    4 days ago
  •  ...Agency is seeking someone to create and execute role-play-based evaluation scenarios that simulate realistic customer service interactions for advanced agentic audio models. You will help develop diverse and representative datasets and assess model performance using standardized... 

    Agency.com

    New York, NY
    1 day ago
  • $110k - $138k

     ...doing? The Computer Vision Specialist is a member of WWT’s Global Solutions...  ...team, part of AI & Data Solutions, focused on computer...  ...Physical AI sub team, the data science and edge AI specialists,...  ...data, and on how models will be evaluated, optimized for edge deployment... 
    Full time
    Work experience placement
    Remote work
    Flexible hours
    Shift work

    World Wide Technology

    New York, NY
    2 days ago
  • Magnetto is seeking a Telegram Ads Specialist with strong analytical skills to lead campaigns across Telegram. You will own the full cycle from launch to optimization and reporting, driving data-driven decisions and improvements. Remote work is available with compensation... 
    Remote job

    Magnetto

    New York, NY
    4 days ago
  • ## Data Labeling AssociateApplylocations: CA Client Field Office (San Fran Bay Area): New...  ...rating guidelines.**Model Training and Evaluation:*** Initiate model training processes using...  ...:*** Bachelor's degree in Computer Science, Data Science, Linguistics or Computational... 
    Full time
    Work at office
    Remote work

    Welocalize

    New York, NY
    18 hours ago
  • Braintrust's client is seeking nutrition and dietary-data professionals to support an AI training and evaluation initiative. You will review food, dietary-pattern, and nutrition information to help ensure scientific accuracy, safety, and usefulness across varied contexts... 

    Braintrust

    New York, NY
    2 days ago
  • Product Hackers busca un Digital Analyst remoto para liderar la estrategia de experimentación y análisis de datos en cuentas de gran volumen, apoyándote en Adobe (Target, Analytics y Launch). Facilitarás decisiones de negocio por medio de insights y gestión de tests, manteniendo...
    Remote job

    Product Hackers

    New York, NY
    2 days ago
  •  ...Data Analyst / Security Master & Reference Data Specialist Location: New York, NY Schedule: Fully On-Site Our client a leading global investment firm...  ...into technology requirements Assist with the evaluation and implementation of new data sources, vendors, and... 
    Work at office

    Phyton Talent Advisors

    New York, NY
    a month ago
  • Crossing Hurdles in the United Kingdom seeks a finance and AI governance specialist to develop realistic, multi-step finance and business process prompts mirroring real industry workflows. You will create objective rubrics to assess AI outputs, review responses using these... 

    Crossing Hurdles

    New York, NY
    2 days ago
  • $80k - $90k

     ...Senior Specialist, Ad Operations & Measurement New York, NY; Remote...  ...created a world where art and science are best friends Known is what happens when a data science firm and a creative agency...  ...internal standards. Regularly evaluate and optimize workflows and... 
    Remote work

    Known

    New York, NY
    4 days ago
  • AI Trainer Jobs is seeking a Swiss German Audio transcription Expert for a remote evaluation track. Review prompts and responses against AuraOne's quality rubric, label edge cases, and provide structured feedback for retraining. You will compare paired outputs, judge voice... 
    Remote job
    Hourly pay
    For contractors

    Auraone

    New York, NY
    18 hours ago
  • Role: English Speech & Audio Evaluation Specialist Employment Type: 3-Month Contractual Role Roles and Responsibilities Listen to English audio recordings and conversations and evaluate the quality of spoken English. Assess pronunciation, accent, fluency, clarity, tone... 

    Innodata India

    New York, NY
    8 hours ago
  • AuraOne, Inc. seeks a Hindi Bilingual Expert for a remote evaluation track to review prompts and responses against the company quality rubric. You...  ...requires clear written reasoning and async availability for at least 10 hours per week. #J-18808-Ljbffr AuraOne Human Data
    Remote job
    10 hours per week

    AuraOne Human Data

    New York, NY
    18 hours ago
  • AuraOne Human Data is seeking a Japanese Bilingual Expert for a remote evaluation track. You will review prompts and responses against AuraOne's quality rubric, label edge cases, and provide structured feedback to retrain models. This contractor role requires strong attention... 
    Remote job
    For contractors

    AuraOne, Inc.

    New York, NY
    8 hours ago
  • AuraOne is seeking a Portuguese Language Expert to remotely evaluate Portuguese generalist prompts and responses. You will review outputs against...  ...feedback the modeling team can use for retraining. As an AI data reviewer, you will help convert outputs into auditable labels... 
    Remote job
    For contractors

    Auraone

    New York, NY
    8 hours ago
  • AuraOne is seeking a French Audio Recording Expert (Canada) for remote evaluation. You will review prompts and responses, compare outputs, and label issues...  ...quality, and conversation robustness for high-quality data labeling. Role requires prior evaluation or annotation... 
    Remote job

    Auraone

    New York, NY
    4 days ago
  • AuraOne is seeking a Galician Bilingual Expert to remotely review prompts and responses against its quality rubric. Reviewers compare...  ...independent contractor, you will help turn Galician bilingual evaluation results into auditable labels, rationales, and regression cases... 
    Remote job
    Hourly pay
    For contractors

    Auraone

    New York, NY
    3 days ago
  •  ...Strong analytical and writing skills — able to define precise, low-variance grading criteria Comfortable working with AI outputs and evaluating them critically and objectively Prior experience in eval design, QA, tutoring, or curriculum/rubric design is a plus Core... 
    Weekly pay
    Remote work
    Home office

    Verita Ai

    New York, NY
    18 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Science Expert - Evaluation Specialist. Be the first to apply!