Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Data Scientist - AI Evaluation

$100 - $150 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Data Scientist Talent Network
Type: Contract
Compensation: $100–$150/hour
Location: Remote

Role Responsibilities

  • Design precise, task-specific grading criteria for data science deliverables. This includes exploratory data analyses , statistical modeling work , machine learning pipelines , and technical reports.
  • Evaluate AI-generated or human-created work against established criteria to ensure quality and accuracy.
  • Provide detailed written justifications for evaluations and scores to maintain transparency.
  • Apply consistent, evidence-based judgment for reproducible and defensible assessments.
  • Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.

Qualifications

Must-Have

  • 1+ years of professional data science experience.
  • Experience at a leading technology, research, or quantitative firm.
  • Strong command of Python , SQL , statistical modeling , and machine learning .
  • Exceptional written communication skills for conveying technical findings.
  • Detail-oriented and consistent approach to evaluating complex work.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 3 days ago
Similar jobs that could be interesting for youBased on the Data Scientist - AI Evaluation in San Francisco, CA vacancy
  •  ...superintelligence stack: the infrastructure frontier AI labs build internally, made available to...  ..., Lab, unifies compute, environments, evaluations, secure sandboxes, high-performance...  ...to turn their own workflows, tools, data, and feedback loops into superintelligence... 
    Suggested
    Full time
    Work at office
    Remote work
    Visa sponsorship
    Relocation package
    Flexible hours

    PrimeIntellect

    San Francisco, CA
    6 days ago
  • $100 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Summers , and Jack Dorsey . Position: Data Scientist Talent Network Type: Contract...  ...pipelines , and A/B test write-ups . Evaluate AI-generated or human-created work... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  • $80 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Summers , and Jack Dorsey . Position: Data Engineer (Coding Agent Experience) Type:...  ...frontier AI coding agents to complete and evaluate complex data engineering tasks. Review... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  •  ...diagnostics by bringing frontier AI into clinical practice. We're tackling...  ...enriched by real-world clinical data. Together, we're building a future...  ...Research Engineer or Research Scientist to advance how we curate data and evaluate frontier AI models for medical imaging... 
    Suggested
    Full time

    epsilon-health

    San Francisco, CA
    6 days ago
  •  ...built world At Bedrock, we're moving AI out of the lab and onto real job sites....  ...you join us. Machine Learning Engineer: Evaluation Bedrock is bringing autonomy to the construction...  ...hardware in the loop systems, and field data from Bedrock Operator equipped machinery.... 
    Suggested
    Full time
    For contractors

    bedrock-robotics

    San Francisco, CA
    6 days ago
  •  ...Data Scientist Our client is seeking a highly skilled Data Scientist to leverage large volumes...  ...data-driven solutions. Continuously evaluate and identify new technologies, tools,...  ...understanding of data science, machine learning, AI, and advanced analytics techniques.... 
    Remote work
    Flexible hours

    Mhymatch Inc

    San Francisco, CA
    17 hours ago
  •  ...Built World At Bedrock, we're moving AI out of the lab and into the real world....  ...The Role Bedrock Robotics is hiring a Data Scientist to lead high-impact data science work across...  ...and product outcomes Design and evaluate experiments, analyses, and validation approaches... 

    Bedrock Robotics

    San Francisco, CA
    4 days ago
  • $180k - $230k

     ...severe, complex diseases. Our data platform is used by drug developers...  ...in bioinformatics, AI, and drug development. We have...  ...are looking for a Senior Data Scientist who will own some of the most...  ...including prompt engineering, model evaluation, and ongoing performance... 
    Work experience placement
    Work at office
    Flexible hours

    Probably Genetic

    San Francisco, CA
    17 hours ago
  • $124k - $329.2k

     ...experienced professionals to elevate our data and analytics efforts. As a Senior Data Scientist in Data Science, you will...  ...expertise and knowledge of data science, AI/ML, analytics engineering, and...  ...experiments, analyze results, evaluating the methodology and performance... 
    Ongoing contract
    Remote work

    GitHub

    San Francisco, CA
    3 days ago
  • $150k - $170k

     ...highly motivated, mission-oriented Applied AI Engineer to help develop, integrate, and...  ...role, you will work closely with other data scientists, software engineers, product teams, and...  ...model integration, inference services, evaluation pipelines, and production-facing AI capabilities... 
    Work experience placement
    Casual work
    Work at office
    Relocation package
    2 days per week

    CHAOS Industries

    San Francisco, CA
    3 days ago
  • $206.1k - $269.08k

     ...Principal Data Scientist Analytics & Data Science | San Francisco, United States | Remote,...  ...experimentation, deployment, and ongoing evaluation Partner with Product, Engineering, and...  ...engineering, model monitoring, and responsible AI/ML across teams At Atlassian, we... 
    Local area
    Remote work

    Trello

    San Francisco, CA
    2 days ago
  • $121.5k - $270k

     ...Navan is looking for a hands-on Senior Data Scientist to apply statistical modeling and AI to business optimization and better decision-making. You’ll partner...  ...that help teams understand customer behavior, evaluate initiatives, and choose where to invest. Build reusable... 

    Navan

    San Francisco, CA
    17 hours ago
  • $226k - $254k

     ...real-customer impact. About the Kikoff data science team Our job is to make sure...  .... This role sits with the data scientists embedded in Kikoff's core credit-building...  ...Kikoff: how we do experimentation, how we evaluate AI products, how we review each other's work... 
    Internship
    Local area

    Kikoff

    San Francisco, CA
    12 hours ago
  •  ...Manager of Data Science for Financial Engineering OpenAI's Financial...  ...a team of high-impact data scientists. Set the technical...  ...modeling, CUPED, or counterfactual evaluation Experience with global payment...  .... OpenAI is an AI research and deployment company... 
    Work at office
    Relocation package
    3 days per week

    OpenAI

    San Francisco, CA
    17 hours ago
  • $152.5k - $205k

     ...Senior Data Scientist On Fraud Analytics Circle is committed to visibility and stability in...  ...Compliance. What you'll do: Build and evaluate fraud models for onboarding, login,...  ...Circle's fraud platform. Help build AI agent workflows for transaction review,... 
    Flexible hours
    Shift work

    Circle

    San Francisco, CA
    17 hours ago
  •  ...Join us to design and deliver AI-powered learning tools that scale...  .... We partner closely with the Data Platform team, who manage our...  ...the Role: As the Data Scientist for Content Platform & Experience...  ...experiments, search and recommendation evaluation, and measure how well AI Coach... 
    Contract work
    Work at office
    2 days per week

    Quizlet

    San Francisco, CA
    17 hours ago
  •  ...AfterQuery is an applied research lab curating data solutions for foundation model development. We serve every frontier AI lab with the mission of delivering the best...  ...Overview AfterQuery builds the data and evaluation systems that power frontier AI models. Every... 
    Full time
    Local area

    AfterQuery

    San Francisco, CA
    6 days ago
  • $148.5k - $223.9k

     ...DetailsAbout SalesforceSalesforce is the #1 AI CRM, where humans with agents drive...  ...platform for Salesforce customers. As a Data Scientist, you will partner with Product, Infrastructure...  ...tools to help our recruiters assess and evaluate candidates’ resumes and qualifications... 
    Full time

    Salesforce

    San Francisco, CA
    4 hours ago
  •  ...team is looking for an experienced Senior Data Scientist to lead brand analytics at DoorDash....  ...reports, and dashboards using cutting-edge AI tooling, SQL, R, Python, or other...  ...audiences, investment, and channel mix, and evaluating the relationship between brand health and... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    4 days ago
  •  ...flexible work model, whereas other Data Science roles may be aligned...  ...now part of Superhuman, the AI productivity platform on a...  ...delights its users? This Data Scientist position is embedded in the Superhuman...  ...data-driven decisions.You’ve evaluated the quality of AI, LLM, or... 
    Remote work
    Worldwide
    Home office
    Flexible hours

    Superhuman

    San Francisco, CA
    12 hours ago
  • $266k - $385k

    About the RoleWe’re hiring a Data Scientist to support Real Estate & Workplace (REW), a fast-moving...  ...execution. Working at the forefront of AI-native analytics, you’ll help define the...  ...inference, forecasting, and model evaluation; fluent in SQL and Python.Proven ability... 
    Work at office
    Local area
    Immediate start
    Flexible hours

    OpenAI

    San Francisco, CA
    12 hours ago
  • $206.6k - $229k

     ...perfect customer experiences.Our AI Agent Fin is the highest-...  ...opportunity?The Research, Analytics & Data Science (RAD) team at Intercom...  ...-making. We're a team of data scientists and product researchers who...  ....Experience with AI/ML evaluation, LLM-driven applications, or conversational... 
    Work at office
    Local area
    Work from home
    Shift work
    3 days per week

    Intercom

    San Francisco, CA
    12 hours ago
  • $347k

     ...QualificationsExperience independently leading data science work on a product, growth, or...  ...customer selection affects the conclusion and evaluate a forecast against a simple baseline....  ...infrastructure investment.About OpenAIOpenAI is an AI research and deployment company dedicated... 
    Work at office
    Local area
    Flexible hours
    Shift work

    OpenAI

    San Francisco, CA
    1 day ago
  • $168.35k - $230.91k

     ...the best chapter of your career. Senior Data Scientist, Forward DeployedForward Deployed Engineering...  ...), Predictive track, in Data Science & AI EngineeringREPORTS TODirector of Data...  ...as XGBoost and Random Forest, model evaluation and calibration.Fluent in Python and SQL... 
    Full time

    6sense

    San Francisco, CA
    1 day ago
  • $150k - $185k

     ...Francisco, CaliforniaEngineering - Data Science & Analytics /US Full-...  ...looking for a hands-on Data Scientist to own and deliver complex...  ...-quality Python, building and evaluating models on large-scale viewership...  ...use artificial intelligence (AI) tools to support parts of the... 
    Full time
    Temporary work

    Samba TV

    San Francisco, CA
    3 days ago
  • $130.1k - $165k

     ...for a problem solver who thrives at the intersection of data science, growth strategy, and AI-powered automation. You’ll focus on diving deep into...  ...experimentation workflows, and surface intelligent recommendations.Evaluate and implement advanced statistical techniques -... 
    Full time
    Temporary work
    Work at office
    Local area
    Relocation package
    Flexible hours
    2 days per week

    Tubi TV

    San Francisco, CA
    2 days ago
  • $139.76k - $287.75k

     ...career you love? It’s Possible.At Pinterest, AI isn't just a feature, it's a powerful...  ...here.We are looking for a Senior Data Scientist to join our marketing org. As a Senior Data...  ...experimentation capabilities and tools to evaluate the business impact of performance marketing... 
    Work at office
    Local area
    Relocation
    Relocation package

    Pinterest

    San Francisco, CA
    1 day ago
  •  ...Analytics team is looking for experienced Data Scientists and Senior Data Scientists to guide...  ...data-supported wayNotice Regarding Use of AI and Automated Tools: To streamline our hiring...  ...: Gem assists our recruiting team by evaluating job related qualifications and... 
    Hourly pay
    Work at office
    Local area
    Remote work
    Flexible hours

    Doordash

    San Francisco, CA
    2 days ago
  • $122k - $240.5k

     ...future of Deloitte's business through new AI-native platforms and products. The team...  ...both! We are looking for a talented Data Scientist to join the Converge for FSI team and help...  ...structured financial services data sources.Evaluate model performance against business and... 
    Local area

    Deloitte

    San Francisco, CA
    1 day ago
  • $190k - $250k

     ...choice and control.We're looking for a Data Scientist to help our GTM and Marketing teams understand...  ...tools that teams rely on.Design and evaluate experiments to understand what's working...  ....GitHub with CI/CD for version control.AI tools (e.g., Claude, ChatGPT, Copilot) are... 

    Thatch Health

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Data Scientist - AI Evaluation. Be the first to apply!