Data Scientist - AI Evaluation
$100 - $150 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Data Scientist Talent Network
Type: Contract
Compensation: $100–$150/hour
Location: Remote
Role Responsibilities
- Design precise, task-specific grading criteria for data science deliverables. This includes exploratory data analyses , statistical modeling work , machine learning pipelines , and technical reports.
- Evaluate AI-generated or human-created work against established criteria to ensure quality and accuracy.
- Provide detailed written justifications for evaluations and scores to maintain transparency.
- Apply consistent, evidence-based judgment for reproducible and defensible assessments.
- Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.
Qualifications
Must-Have
- 1+ years of professional data science experience.
- Experience at a leading technology, research, or quantitative firm.
- Strong command of Python , SQL , statistical modeling , and machine learning .
- Exceptional written communication skills for conveying technical findings.
- Detail-oriented and consistent approach to evaluating complex work.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
- ...superintelligence stack: the infrastructure frontier AI labs build internally, made available to... ..., Lab, unifies compute, environments, evaluations, secure sandboxes, high-performance... ...to turn their own workflows, tools, data, and feedback loops into superintelligence...SuggestedFull timeWork at officeRemote workVisa sponsorshipRelocation packageFlexible hours
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Scientist Talent Network Type: Contract... ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work...SuggestedContract workSummer workRemote work$80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Summers , and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type:... ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review...SuggestedContract workSummer workRemote work- ...diagnostics by bringing frontier AI into clinical practice. We're tackling... ...enriched by real-world clinical data. Together, we're building a future... ...Research Engineer or Research Scientist to advance how we curate data and evaluate frontier AI models for medical imaging...SuggestedFull time
- ...built world At Bedrock, we're moving AI out of the lab and onto real job sites.... ...you join us. Machine Learning Engineer: Evaluation Bedrock is bringing autonomy to the construction... ...hardware in the loop systems, and field data from Bedrock Operator equipped machinery....SuggestedFull timeFor contractors
- ...Data Scientist Our client is seeking a highly skilled Data Scientist to leverage large volumes... ...data-driven solutions. Continuously evaluate and identify new technologies, tools,... ...understanding of data science, machine learning, AI, and advanced analytics techniques....Remote workFlexible hours
- ...Built World At Bedrock, we're moving AI out of the lab and into the real world.... ...The Role Bedrock Robotics is hiring a Data Scientist to lead high-impact data science work across... ...and product outcomes Design and evaluate experiments, analyses, and validation approaches...
$180k - $230k
...severe, complex diseases. Our data platform is used by drug developers... ...in bioinformatics, AI, and drug development. We have... ...are looking for a Senior Data Scientist who will own some of the most... ...including prompt engineering, model evaluation, and ongoing performance...Work experience placementWork at officeFlexible hours$124k - $329.2k
...experienced professionals to elevate our data and analytics efforts. As a Senior Data Scientist in Data Science, you will... ...expertise and knowledge of data science, AI/ML, analytics engineering, and... ...experiments, analyze results, evaluating the methodology and performance...Ongoing contractRemote work$150k - $170k
...highly motivated, mission-oriented Applied AI Engineer to help develop, integrate, and... ...role, you will work closely with other data scientists, software engineers, product teams, and... ...model integration, inference services, evaluation pipelines, and production-facing AI capabilities...Work experience placementCasual workWork at officeRelocation package2 days per week$206.1k - $269.08k
...Principal Data Scientist Analytics & Data Science | San Francisco, United States | Remote,... ...experimentation, deployment, and ongoing evaluation Partner with Product, Engineering, and... ...engineering, model monitoring, and responsible AI/ML across teams At Atlassian, we...Local areaRemote work$121.5k - $270k
...Navan is looking for a hands-on Senior Data Scientist to apply statistical modeling and AI to business optimization and better decision-making. You’ll partner... ...that help teams understand customer behavior, evaluate initiatives, and choose where to invest. Build reusable...$226k - $254k
...real-customer impact. About the Kikoff data science team Our job is to make sure... .... This role sits with the data scientists embedded in Kikoff's core credit-building... ...Kikoff: how we do experimentation, how we evaluate AI products, how we review each other's work...InternshipLocal area- ...Manager of Data Science for Financial Engineering OpenAI's Financial... ...a team of high-impact data scientists. Set the technical... ...modeling, CUPED, or counterfactual evaluation Experience with global payment... .... OpenAI is an AI research and deployment company...Work at officeRelocation package3 days per week
$152.5k - $205k
...Senior Data Scientist On Fraud Analytics Circle is committed to visibility and stability in... ...Compliance. What you'll do: Build and evaluate fraud models for onboarding, login,... ...Circle's fraud platform. Help build AI agent workflows for transaction review,...Flexible hoursShift work- ...Join us to design and deliver AI-powered learning tools that scale... .... We partner closely with the Data Platform team, who manage our... ...the Role: As the Data Scientist for Content Platform & Experience... ...experiments, search and recommendation evaluation, and measure how well AI Coach...Contract workWork at office2 days per week
- ...AfterQuery is an applied research lab curating data solutions for foundation model development. We serve every frontier AI lab with the mission of delivering the best... ...Overview AfterQuery builds the data and evaluation systems that power frontier AI models. Every...Full timeLocal area
$148.5k - $223.9k
...DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive... ...platform for Salesforce customers. As a Data Scientist, you will partner with Product, Infrastructure... ...tools to help our recruiters assess and evaluate candidates’ resumes and qualifications...Full time- ...team is looking for an experienced Senior Data Scientist to lead brand analytics at DoorDash.... ...reports, and dashboards using cutting-edge AI tooling, SQL, R, Python, or other... ...audiences, investment, and channel mix, and evaluating the relationship between brand health and...Hourly payWork at officeLocal areaRemote workFlexible hours
- ...flexible work model, whereas other Data Science roles may be aligned... ...now part of Superhuman, the AI productivity platform on a... ...delights its users? This Data Scientist position is embedded in the Superhuman... ...data-driven decisions.You’ve evaluated the quality of AI, LLM, or...Remote workWorldwideHome officeFlexible hours
$266k - $385k
About the RoleWe’re hiring a Data Scientist to support Real Estate & Workplace (REW), a fast-moving... ...execution. Working at the forefront of AI-native analytics, you’ll help define the... ...inference, forecasting, and model evaluation; fluent in SQL and Python.Proven ability...Work at officeLocal areaImmediate startFlexible hours$206.6k - $229k
...perfect customer experiences.Our AI Agent Fin is the highest-... ...opportunity?The Research, Analytics & Data Science (RAD) team at Intercom... ...-making. We're a team of data scientists and product researchers who... ....Experience with AI/ML evaluation, LLM-driven applications, or conversational...Work at officeLocal areaWork from homeShift work3 days per week$347k
...QualificationsExperience independently leading data science work on a product, growth, or... ...customer selection affects the conclusion and evaluate a forecast against a simple baseline.... ...infrastructure investment.About OpenAIOpenAI is an AI research and deployment company dedicated...Work at officeLocal areaFlexible hoursShift work$168.35k - $230.91k
...the best chapter of your career. Senior Data Scientist, Forward DeployedForward Deployed Engineering... ...), Predictive track, in Data Science & AI EngineeringREPORTS TODirector of Data... ...as XGBoost and Random Forest, model evaluation and calibration.Fluent in Python and SQL...Full time$150k - $185k
...Francisco, CaliforniaEngineering - Data Science & Analytics /US Full-... ...looking for a hands-on Data Scientist to own and deliver complex... ...-quality Python, building and evaluating models on large-scale viewership... ...use artificial intelligence (AI) tools to support parts of the...Full timeTemporary work$130.1k - $165k
...for a problem solver who thrives at the intersection of data science, growth strategy, and AI-powered automation. You’ll focus on diving deep into... ...experimentation workflows, and surface intelligent recommendations.Evaluate and implement advanced statistical techniques -...Full timeTemporary workWork at officeLocal areaRelocation packageFlexible hours2 days per week$139.76k - $287.75k
...career you love? It’s Possible.At Pinterest, AI isn't just a feature, it's a powerful... ...here.We are looking for a Senior Data Scientist to join our marketing org. As a Senior Data... ...experimentation capabilities and tools to evaluate the business impact of performance marketing...Work at officeLocal areaRelocationRelocation package- ...Analytics team is looking for experienced Data Scientists and Senior Data Scientists to guide... ...data-supported wayNotice Regarding Use of AI and Automated Tools: To streamline our hiring... ...: Gem assists our recruiting team by evaluating job related qualifications and...Hourly payWork at officeLocal areaRemote workFlexible hours
$122k - $240.5k
...future of Deloitte's business through new AI-native platforms and products. The team... ...both! We are looking for a talented Data Scientist to join the Converge for FSI team and help... ...structured financial services data sources.Evaluate model performance against business and...Local area$190k - $250k
...choice and control.We're looking for a Data Scientist to help our GTM and Marketing teams understand... ...tools that teams rely on.Design and evaluate experiments to understand what's working... ....GitHub with CI/CD for version control.AI tools (e.g., Claude, ChatGPT, Copilot) are...
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Data Scientist - AI Evaluation. Be the first to apply!
- energy data scientist San Francisco, CA
- entry level data scientist San Francisco, CA
- data scientist (hedge fund) San Francisco, CA
- python data scientist (contract) San Francisco, CA
- senior data scientist San Francisco, CA
- data scientist San Francisco, CA
- python data scientist San Francisco, CA
- ai data scientist San Francisco, CA
- work from home data scientist San Francisco, CA
- junior data scientist remote San Francisco, CA



