Data Science Expert - AI Evaluation
$120 - $170 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Science Expert
Type: Contract
Compensation: $120–$170/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for real-world data science deliverables such as analyses, models, dashboards, and experiment readouts.
- Score AI-generated and human work samples against criteria with detailed written justifications for every score.
- Apply consistent, evidence-based judgment to ensure scores are reproducible and defensible.
- Incorporate structured feedback from senior reviewers and iterate quickly on your work.
- Work independently and asynchronously to meet deadlines while improving AI model performance .
Qualifications
Must-Have
- 5+ years of professional data science experience in industry.
- Background in business operations, product, or growth data science at top-tier technology companies.
- Deep fluency in experiment design and A/B testing, metric definition, SQL/Python analysis, and communicating findings to executive stakeholders.
- Exceptionally strong written communication.
- Detail-oriented, consistent, and comfortable having your judgment reviewed and calibrated against peers.
Preferred
- Prior experience with AI training , evaluation, or human-data projects.
Application Process (Takes 20–30 mins to complete)
- Submit your resume or relevant technical background to get started.
- Qualified applicants may be asked to complete a brief technical assessment or submit additional information.
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on us.fitly.work
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 5 days ago
Similar jobs that could be interesting for youBased on the Data Science Expert - AI Evaluation in New York, NY vacancy
- Data Science Expert - AI Content Specialist What if your expertise in machine learning and statistical modeling could directly shape how the world... ...benchmark for AI responses Audit AI-Generated Code — Evaluate outputs from models using Scikit-Learn, PyTorch, TensorFlow...SuggestedHourly payOngoing contractContract workFreelanceRemote workFlexible hours
$120 - $170 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Science Expert Type: Contract Compensation:... ...Prior experience with AI training , evaluation, or human-data projects....SuggestedContract workSummer workRemote work- ...Job Description Job Description Job Title: AI Data Science Domain Expert Job Type: Contractor (Part-Time) Location: Remote Job Overview... ...-generation AI systems. In this role, you will review, evaluate, and refine AI-generated technical and analytical content...SuggestedRemote jobPart timeFor contractors
$60 - $90 per hour
...and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Data Science & Quantitative Analysis Expert Type: Contract Compensation... ...provide clear recommendations. Evaluate model performance on tasks, ensuring...SuggestedContract workSummer workRemote work- Welo Data is seeking a Data Labeling Associate in New York City to evaluate AI systems, focusing on data quality and model behavior. This full-time position requires native British English proficiency and a bachelor's degree. You'll work closely with AI, providing critical...SuggestedFull time
$61k - $101k
...require a masters degree in Science, Technology,... ...in Python and SQL for data handling and analysis.... ...Models (LLMs) and agentic AI approaches. We need... ...Preferred: familiarity with evaluation and monitoring concepts... ...working with subject matter experts to align them with...Full timeJ.P. Morgan
New York, NY5 days ago$145k - $160k
...aren’t just advisors we’re doers. We use AI tools to accelerate workflows and deliver... ...Business Analyst to work on client-facing data and AI platform delivery. The role owns... ...for predictive and agentic capabilities: evaluation criteria, ground truth sets, acceptable error...Full timeSol Consulting
New York, NY26 days ago- ...About the role We are hiring expert Evaluators in Data analysis / quantitative readouts to review and assess AI-generated work products (documents, spreadsheets, and slide decks) for accuracy, rigor, and domain quality. You will apply deep subject-matter expertise...Hourly payWork at officeRemote work
Mercor Inc
New York, NY2 days ago $63k - $141.5k
...SummaryThe Opportunity As part of the AI Engineering team within the... ...to design and develop robust data solutions for clients. Within... ...involving testing and evaluation of AI-enabled systems, analyze... ...fields of study: Analytics/Data Science, Artificial Intelligence/Robotics...Full timeH1bPwC
New York, NY1 day ago$70.72k
...Overview Welo Data is looking for sharp, curious, and detail-oriented individuals to join our team as Data Labeling Associate... ...role. You’ll be working directly with cutting-edge AI systems — evaluating outputs, identifying gaps, and helping improve how these systems...Full timeContract workRemote workVisa sponsorship$70.72k
\n What You’ll Do Evaluate AI model outputs and provide structured, high-quality feedback Perform audit-based reviews of data and model behavior — identifying patterns, edge cases, and failure modes Apply guidelines thoughtfully — and flag when they don’...Full time$195k - $275k
...industry experience to join a data analytics and artificial intelligence... ...capabilities in quantitative AI, combining advanced... ...statistical approaches to support evaluation, monitoring, and continuous improvement... ...Statistics, Physics, Computer Science, Engineering or related...Full timeTemporary workMorgan Stanley
New York, NYmore than 2 months ago$96.89k - $155.02k
...ones Capital Group Rotational Program Data & Technology Track Associate - US - New... ...-making and business outcomes. Data Science: Apply advanced analytics, modeling, artificial... ...and visualizations, support data and AI initiatives, evaluate business requirements, and help create...Rotational programFull timeTemporary workInternshipLocal areaFlexible hoursCapital Group
New York, NY2 days ago$38 per hour
Data Quality Analyst Welo Data is looking for experienced, detail-oriented professionals... ...You'll work closely with both people and AI systems — auditing outputs, supporting... ...AI/ML data workflows or human-in-the-loop evaluation environments Prior experience auditing or...Full timeContract workRemote workVisa sponsorship- AI Engineer - NLP/LLM Data Specialist Location: Houston, Texas - Remote Duration: 6 months About the Role We are growing our Data Science and Data Engineering team and are looking for an experienced... ...objectives. Technology Selection Evaluate and recommend AI technologies,...Remote work
- ...A tech and AI-focused referral partner is seeking a SaaS Software Data Reviewer to analyze data from B2B tools and create prompts based on SaaS ticket content. The job requires strong writing skills, proficiency in task management tools like Jira and Slack, and attention...Hourly payContract workRemote work
Crossing Hurdles
New York, NY2 days ago - A leading AI company is looking for a Remote Business Analyst who excels in analytical tasks and is fluent in both English and Japanese. This role involves answering analytical questions and working with large language models to improve performance. Candidates should possess...Remote job
$90k - $112k
Jobright.ai is seeking a Junior Data Analyst to support translational research at Columbia University Irving Medical Center. The role involves designing assays, performing statistical analysis, and providing bioinformatic support. Candidates should have a Bachelor’s degree...- Transforming data into actionable insights, the full-time remote Data and AI Analyst will analyze data, develop reporting solutions, and leverage AI tools to drive... ..., Mathematics, Economics, Engineering, Computer Science, Business Analytics, or a related quantitative field...Full timeRemote work
$26 - $28 per hour
...Description OverviewWelo Data is looking for detail-oriented and reliable individuals... ...Analysts, supporting speech and voice AI systems.This is a high-impact production... ...this role is more execution-focused than evaluation-heavy roles, it still requires strong judgment...Full timeRemote workVisa sponsorship- ...We are seeking a seasoned Data Technology Consultant to join our expanding Data & AI practice and help drive Data & AI Modernization Programs for EPAM clients... ...seamlessly across sales, delivery, and technical expert teams to ensure proposed solutions perfectly match...Temporary workWorldwide
EPAM Systems
New York, NY13 days ago - Welo Data is hiring Data Labeling Associates in New York City for Project Perseus, requiring professional-level Turkish proficiency... .... This role blends human judgment and machine learning to evaluate Arabic AI outputs, focusing on safety and quality. Ideal candidates...Full time
- ...Home Depot is redefining how talent intelligence, technology, and data come together to shape the future of hiring. As a Senior Analyst,... ...of talent acquisition, workforce strategy, analytics, and AI, transforming talent data into actionable intelligence that helps...
$85k - $110k
..., Inc is looking for an Insights team member to analyze customer data and deliver actionable insights. With a competitive salary range... ...000, this role offers growth opportunities within a fast-growing AI company. You will partner with internal teams and support customer...- YO AI Labs seeks a Data Analyst contractor to support a customer project focusing on business data analysis, document review, privacy, and QA for AI training. No prior AI experience is required. Responsibilities include reviewing datasets for accuracy, performing document...Remote jobContract workFor contractors
- About the role We are hiring expert Evaluators in Data Science to review and assess AI-generated work products (slides, spreadsheets, and documents) for real-world quality. You will apply deep subject-matter expertise to grade outputs. This is a remote, hourly engagement...Remote jobHourly payWork at office
- ...Overview We are looking for Generalists with strong analytical and critical-thinking skills to support AI and data-focused projects. The role involves reviewing, evaluating, and annotating information across different topics while maintaining high accuracy and quality....Remote jobFor contractors
$150 per hour
...technical talent with leading AI research labs. Headquartered in... ...Jack Dorsey . Position: Data Scientist Talent Network Type... ...grading criteria for data science deliverables. This includes exploratory... ..., and technical reports. Evaluate AI-generated or human-...Remote jobContract workSummer work- ...Software Engineers to support an AI training project by creating... ...learning environments that evaluate AI models on complex software... ...understanding of algorithms and data structures . Experience... ...apply. Availability Selected experts should be prepared to begin...Remote jobFor contractors
- An environmental consulting firm is looking for Environmental Scientists and Specialists to provide expertise in refining AI systems. This role involves analyzing various environmental datasets, contributing to risk assessments, and collaborating with technical teams....Remote jobFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Science Expert - AI Evaluation. Be the first to apply!
Related searches
- data entry associate New York, NY
- data integration specialist New York, NY
- data associate New York, NY
- data operations specialist New York, NY
- provider data specialist New York, NY
- data visualization specialist New York, NY
- oncology data specialist New York, NY
- data science associate New York, NY
- data management specialist New York, NY
- data conversion specialist New York, NY