Data Science Expert - Evaluation Specialist
$120 - $170 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Science Expert
Type: Contract
Compensation: $120–$170/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for real-world data science deliverables, including analyses, models, dashboards, and experiment readouts.
- Score AI-generated and human work samples against established criteria with detailed, well-reasoned written justifications.
- Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate quickly on your work.
- Collaborate with AI research teams to improve training data quality and downstream performance.
Qualifications
Must-Have
- 5+ years of professional data science experience in industry.
- Background in business operations, product, or growth data science at top-tier technology companies.
- Deep fluency in experiment design, A/B testing , metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
- Exceptionally strong written communication.
- Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
Preferred
- Prior experience with AI training , evaluation, or human-data projects.
Application Process (Takes 20–30 mins to complete)
- Submit your resume or relevant technical background to get started.
- Qualified applicants may be asked to complete a brief technical assessment or submit additional information.
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
$120 - $170 per hour
...Angelo , Larry Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation: $120–$170/hour... .... Preferred ~ Prior experience with AI training , evaluation, or human-data projects. Application Process (Takes 20...SuggestedContract workSummer workRemote work$45 - $55 per hour
...the project channel. Qualifications Must-Have Able to commit 30 hours per week . Prior experience with AI data work . Sharp eye for detail and patience to double-check work. Track record in detail-heavy work such as operations accounting...SuggestedFull timeContract workSummer workRemote work- Crewlia is seeking an AI Data Analyst for a remote hourly contract in the United States and Canada. You will assess AI-generated data... ...reasoning, with flexible hours and clear project guidelines. You will evaluate outputs against source materials, documenting rationales and any...SuggestedRemote jobHourly payContract workFlexible hours
- Seoulstart is seeking a Data Analyst (AI Evaluation) to join our remote team in the United States. You will review datasets and outputs for quality, apply evaluation rubrics, and provide clear feedback to improve AI training data. The role emphasizes attention to detail...SuggestedRemote jobHourly pay
$25 - $50 per hour
...collaboration About micro1 micro1 is a data lab that helps AI labs &... ...agents. We provide frontier evaluations and reinforcement learning... ...sources and vets domain experts for our data platform that enables... ...Engineering & physics Earth sciences & GIS Data science & statistics...SuggestedFor contractorsRemote workFlexible hours$50 - $80 per hour
The work You will help evaluate how well AI handles real-world business analysis tasks. The work combines data privacy review, analytical quality checks, and clear scoring standards for AI performance. Review business datasets for personally identifiable information and...Hourly payPart timeFor contractorsRemote work- Turing is seeking a Business Analyst to review AI responses and validate information accuracy for an evaluation project. You will assess AI outputs, perform online research, and provide structured annotations and feedback to improve model performance in reasoning and task...Hourly pay
- Crossing Hurdles is seeking a Business Analyst for a remote, short-term contract. The role involves evaluating chatbot outputs against criteria, conducting structured analyses, and delivering clear assessments to enhance AI response quality. You will collaborate with cross...Remote jobTemporary work
$45 - $55 per hour
...Location: Remote Role Responsibilities Evaluate the accuracy and depth of AI-generated... ...AI research teams to improve training data quality and downstream performance.... ...solutions. Collaborate with subject matter experts to ensure consistency, relevance, and...Contract workSummer workRemote work$20 - $50 per hour
At TELUS Digital, data is the lifeblood of our innovation... ...Architect and execute evaluation frameworks to measure... ...’s Degree in Data Science, Mathematics,... ...Currently, pay rates for experts range from $20 - $50 USD... ...Collaborate with a network of specialists across 20+ domains and...Hourly payFor contractorsRemote workFlexible hours$200k - $300k
...River Trading (HRT) is seeking a Research Data Expert to join our Data team. Data is a... ...with the sourcing team to identify and evaluate new data procurement opportunitiesAnalyze... ...Python/R preferred) and proficiency in data science methodologies5+ years of professional experience...Work at officeLocal areaImmediate start- ...About the role Calibration Technology Evaluation Specialist is a remote review track for evaluating AI outputs across calibration technology evaluation specialist operations workflows. Reviewers grade workflow correctness, policy adherence, and stakeholder fit; flag operational...Hourly payFor contractorsWork experience placementRemote work10 hours per week
$75 - $110 per hour
...Neurology Evaluation Specialist is a remote clinical-review track for evaluating AI outputs that touch neurology. Reviewers grade differential reasoning, dosing logic, and guideline adherence; flag patient-safety issues; and document the corrected clinical reasoning so...Hourly payFor contractorsRemote work10 hours per week- ...Agency is seeking someone to create and execute role-play-based evaluation scenarios that simulate realistic customer service interactions for advanced agentic audio models. You will help develop diverse and representative datasets and assess model performance using standardized...
$110k - $138k
...doing? The Computer Vision Specialist is a member of WWT’s Global Solutions... ...team, part of AI & Data Solutions, focused on computer... ...Physical AI sub team, the data science and edge AI specialists,... ...data, and on how models will be evaluated, optimized for edge deployment...Full timeWork experience placementRemote workFlexible hoursShift work- Magnetto is seeking a Telegram Ads Specialist with strong analytical skills to lead campaigns across Telegram. You will own the full cycle from launch to optimization and reporting, driving data-driven decisions and improvements. Remote work is available with compensation...Remote job
- ## Data Labeling AssociateApplylocations: CA Client Field Office (San Fran Bay Area): New... ...rating guidelines.**Model Training and Evaluation:*** Initiate model training processes using... ...:*** Bachelor's degree in Computer Science, Data Science, Linguistics or Computational...Full timeWork at officeRemote work
- Braintrust's client is seeking nutrition and dietary-data professionals to support an AI training and evaluation initiative. You will review food, dietary-pattern, and nutrition information to help ensure scientific accuracy, safety, and usefulness across varied contexts...
- Product Hackers busca un Digital Analyst remoto para liderar la estrategia de experimentación y análisis de datos en cuentas de gran volumen, apoyándote en Adobe (Target, Analytics y Launch). Facilitarás decisiones de negocio por medio de insights y gestión de tests, manteniendo...Remote job
- ...Data Analyst / Security Master & Reference Data Specialist Location: New York, NY Schedule: Fully On-Site Our client a leading global investment firm... ...into technology requirements Assist with the evaluation and implementation of new data sources, vendors, and...Work at office
- Crossing Hurdles in the United Kingdom seeks a finance and AI governance specialist to develop realistic, multi-step finance and business process prompts mirroring real industry workflows. You will create objective rubrics to assess AI outputs, review responses using these...
$80k - $90k
...Senior Specialist, Ad Operations & Measurement New York, NY; Remote... ...created a world where art and science are best friends Known is what happens when a data science firm and a creative agency... ...internal standards. Regularly evaluate and optimize workflows and...Remote work- AI Trainer Jobs is seeking a Swiss German Audio transcription Expert for a remote evaluation track. Review prompts and responses against AuraOne's quality rubric, label edge cases, and provide structured feedback for retraining. You will compare paired outputs, judge voice...Remote jobHourly payFor contractors
- Role: English Speech & Audio Evaluation Specialist Employment Type: 3-Month Contractual Role Roles and Responsibilities Listen to English audio recordings and conversations and evaluate the quality of spoken English. Assess pronunciation, accent, fluency, clarity, tone...
- AuraOne, Inc. seeks a Hindi Bilingual Expert for a remote evaluation track to review prompts and responses against the company quality rubric. You... ...requires clear written reasoning and async availability for at least 10 hours per week. #J-18808-Ljbffr AuraOne Human DataRemote job10 hours per week
- AuraOne Human Data is seeking a Japanese Bilingual Expert for a remote evaluation track. You will review prompts and responses against AuraOne's quality rubric, label edge cases, and provide structured feedback to retrain models. This contractor role requires strong attention...Remote jobFor contractors
- AuraOne is seeking a Portuguese Language Expert to remotely evaluate Portuguese generalist prompts and responses. You will review outputs against... ...feedback the modeling team can use for retraining. As an AI data reviewer, you will help convert outputs into auditable labels...Remote jobFor contractors
- AuraOne is seeking a French Audio Recording Expert (Canada) for remote evaluation. You will review prompts and responses, compare outputs, and label issues... ...quality, and conversation robustness for high-quality data labeling. Role requires prior evaluation or annotation...Remote job
- AuraOne is seeking a Galician Bilingual Expert to remotely review prompts and responses against its quality rubric. Reviewers compare... ...independent contractor, you will help turn Galician bilingual evaluation results into auditable labels, rationales, and regression cases...Remote jobHourly payFor contractors
- ...Strong analytical and writing skills — able to define precise, low-variance grading criteria Comfortable working with AI outputs and evaluating them critically and objectively Prior experience in eval design, QA, tutoring, or curriculum/rubric design is a plus Core...Weekly payRemote workHome office
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Science Expert - Evaluation Specialist. Be the first to apply!
- data specialist New York, NY
- data operations specialist New York, NY
- data management specialist New York, NY
- data conversion specialist New York, NY
- data visualization specialist New York, NY
- research data specialist New York, NY
- oncology data specialist New York, NY
- data migration specialist New York, NY
- data entry associate New York, NY
- data management associate New York, NY






