Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Retail SME - AI Evaluation Expert

$60 - $80 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: Retail Specialist
Type: Contract
Compensation: $60–$80/hour
Location: Remote

Role Responsibilities

  • Guide research and engineering teams to close knowledge gaps in retail merchandising , category management , and operations reasoning .
  • Design challenging, domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
  • Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
  • Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks.
  • Collaborate with other subject matter experts to ensure consistency and accuracy in training data .

Qualifications

Must-Have

  • 8+ years of dedicated professional experience in retail (e.g., merchandising, category management, retail operations, buying/planning) at a recognized, top-tier organization (e.g., Amazon, Walmart, Target, Nike, Costco, Home Depot, or equivalent).
  • Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria — mandatory; please describe this experience in your application.
  • Demonstrable career progression (e.g., Category Manager → Senior Manager → Director of Merchandising).
  • Ability to engage reliably for at least 35 hours/week during weekdays.
  • Verbal and written communication skills, problem-solving skills, and interpersonal skills.

Compensation & Legal

  • W-2 employment with Cincinnatus LLC

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 4 days ago
Similar jobs that could be interesting for youBased on the Retail SME - AI Evaluation Expert in New York, NY vacancy
  • $60 - $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    15 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  .... Position: User/customer research and feedback synthesis Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    14 days ago
  • $100 - $120 per unit

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Baseball Fan – Live MLB Game AI Evaluator (US) Type: Contract Compensation:...  ..., schedules, broadcasts, highlights, and expert-style analysis. Run the same question... 
    Suggested
    Contract work
    Work experience placement
    Summer work
    Remote work

    Mercor

    New York, NY
    17 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    12 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Suggested
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    3 days ago
  •  ...About the Opportunity Our client, a leading AI research organization, is looking for...  ...and private credit professionals to help evaluate and improve how AI systems analyze investment...  .... Provide structured feedback and expert insights to improve AI systems' ability to... 
    Freelance
    Remote work

    Braintrust

    New York, NY
    1 day ago
  •  ...About the Opportunity Our client, a leading AI research organization, is looking for experienced software engineers and quantitative...  ...developers with strong functional programming expertise to help evaluate and improve how AI systems write, understand, and solve complex... 
    Freelance
    Remote work

    Braintrust

    New York, NY
    1 day ago
  •  ...About the Opportunity Our client, a leading AI research organization, is looking for...  ...quantitative finance professionals to help evaluate and improve how AI systems analyze...  ...outputs. Provide structured feedback and expert insights to improve AI systems' quantitative... 
    Freelance
    Remote work

    Braintrust

    New York, NY
    1 day ago
  • $90 per hour

     ...Help an AI research team evaluate whether a simulated industrial process plant behaves like a real one. You’ll apply hands-on plant reliability experience to review the simulated environment, synthetic sensor data, and proposed equipment fault and maintenance scenarios... 
    Hourly pay
    Contract work
    Freelance
    Remote work

    Braintrust

    New York, NY
    2 days ago
  • $37 per unit

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Location: Remote Role Responsibilities Evaluate the accuracy and depth of AI-generated...  .... Collaborate with subject matter experts to ensure consistency, relevance, and coverage... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    27 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...and Jack Dorsey . Position: Biology / environmental science Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    New York, NY
    3 days ago
  • $65 - $70 per hour

     ...Biology Expert (PhD) — AI Safety is a remote red-team track for stress-testing AI systems against adversarial prompts. Reviewers craft attack...  ...violated so the safety team can patch the gap. Adversarial evaluation is how AuraOne hardens AI models before they ship to... 
    For contractors
    Remote work
    10 hours per week

    Auraone

    New York, NY
    2 days ago
  • Generative Ai Architect, Interior Designer, Or Senior Architectural Visualisation Specialist Join a leading AI research team's Generative...  ...professional design and construction knowledge to train and evaluate next-generation AI systems. You will create realistic... 
    Weekday work

    Weekday

    New York, NY
    2 days ago
  •  ...education and research sector is seeking a Social Sciences PhD Expert for a remote part-time role, requiring strong analytical...  ...The position involves creating historically relevant prompts, evaluating AI outputs, and contributing to research initiatives. Ideal candidates... 
    Part time
    Remote work
    Flexible hours

    Crossing Hurdles

    New York, NY
    4 days ago
  •  ...at the scale of a major financial enterprise and we are building AI agents that serve millions of customers directly. This is not a...  ...hands and feet of every agent you build. Build and tune the evaluation loop automated evals, regression suites, trace analysis, and A/B... 
    For contractors

    Mindlance

    New York, NY
    23 days ago
  • AI Photo Rater Role Type: Contractor Location: United States (Remote) micro1 is engaging...  ...to support a customer's innovative image evaluation project. In this role, you'll apply your...  ...with a remote team of independent experts to address project goals and technical challenges... 
    For contractors
    Remote work

    Micro1

    New York, NY
    2 days ago
  • Location: Remote, United States Contract type Contractor Project: AI image evaluation About the opportunity micro1 is looking for AI Photo Raters to support a customer project focused on evaluating AI-generated images. In this role, you'll use your visual judgment... 
    Contract work
    For contractors
    Remote work

    micro1 Inc.

    New York, NY
    2 days ago
  • Natural Gas Compression Subject Matter Expert (SME) - Remote AECOM is seeking a Natural Gas Compression Subject Matter Expert (SME) to...  ...coolers, scrubbers, gas filtration). Lead high-level Technical Bid Evaluations (TBEs), vendor package integration, factory acceptance... 
    Remote job
    Work at office
    Relocation

    AECOM

    New York, NY
    2 days ago
  • SAP Cloud SME (SAP Cloud Integration / SAP Cloud Analytics) At [Your Company], we specialize...  ...We are seeking a SAP Cloud Subject Matter Expert to join our growing team and lead the...  ...enterprise applications. Familiarity with AI/ML capabilities in SAP Cloud Analytics. Project... 

    Kaav Inc.

    New York, NY
    2 days ago
  • $8 - $65 per hour

     ...Are you an Arabic (Levantine) language expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into powerful...  ...and suggest improvements to our prompt engineering and evaluation metrics. A Master's degree in Linguistics, Middle... 
    Hourly pay
    Contract work
    For contractors
    Freelance
    Remote work

    Agency.com

    New York, NY
    2 days ago
  • $40 per hour

     ...create high-quality STEM training data for frontier AI models. Our data is used directly in training and evaluation pipelines at leading AI labs to improve model reasoning in technical domains. We are looking for experts in Physics to design rigorous, deterministic... 
    Contract work
    Part time
    Remote work

    Anyone AI

    New York, NY
    2 days ago
  • $40 - $125 per hour

     ...Computational Biology & Bioinformatics Expert, AI Trainer (Remote, Flexible Hours)100% Remote | Work When You Want | Part-Time or Full...  ...biologists and bioinformaticians to work as remote AI trainers and evaluate how frontier AI models handle real biological data analysis:... 
    Hourly pay
    Full time
    Contract work
    Temporary work
    Part time
    Freelance
    Remote work
    Work from home
    10 hours per week
    Flexible hours
    Afternoon shift

    Crucial Recruiting

    New York, NY
    a month ago
  • Principal Network Architect / Subject Matter Expert (SME) Valiant Solutions is seeking a Principal Network Architect / Subject Matter Expert...  ...without dropping internet or cloud connectivity. Plan and evaluate failover testing with the security team to confirm that backup... 
    Remote job
    Contract work
    For contractors
    Interim role

    ClearanceJobs

    New York, NY
    2 days ago
  • VA Subject Matter Expert (SME) Team Lead Prometheus Federal Services (PFS) is a trusted partner to federal health agencies. We are seeking...  ...expertise and strategic guidance to support comprehensive evaluations of Women's Health Programs, identification of opportunities... 
    Contract work
    Work at office
    Local area
    Flexible hours

    Prometheus Federal Services

    New York, NY
    2 days ago
  • $80 - $150 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Jack Dorsey . Position: UI / UX Design Expert Type: Contract Compensation: $...  ...Work independently and asynchronously to evaluate and improve AI model performance .... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    10 days ago
  • $60 - $90 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...for sound scientific reasoning versus superficial reasoning. Evaluate models by reviewing attempts and identifying immediate mistakes.... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    New York, NY
    15 days ago
  • $65 - $75 per unit

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Position: Nuclear Engineering & Safeguards Experts for Red Team Type: Contract Compensation...  ...benign, dual-use, and adversarial. Evaluate AI model responses against a defined... 
    Contract work
    Summer work
    Remote work
    Flexible hours

    Mercor

    New York, NY
    4 days ago
  • $85 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Responsibilities Use frontier AI coding agents to complete and evaluate complex machine learning and AI engineering tasks. Review... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    14 days ago
  • $70 - $100 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Computational Particle & Nuclear Physics Expert Type: Contract Compensation: $...  ...tools. Familiarity with benchmark or evaluation design. Background in scientific teaching... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    4 days ago
  • $80 - $160 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Summers , and Jack Dorsey . Position: Retail / Clienteling professionals Type: Contract...  .... Collaborate with teams to enhance evaluation projects using strong analytical skills.... 
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Retail SME - AI Evaluation Expert. Be the first to apply!