Retail SME - AI Evaluation Expert
$60 - $80 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Retail Specialist
Type: Contract
Compensation: $60–$80/hour
Location: Remote
Role Responsibilities
- Guide research and engineering teams to close knowledge gaps in retail merchandising , category management , and operations reasoning .
- Design challenging, domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
- Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
- Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks.
- Collaborate with other subject matter experts to ensure consistency and accuracy in training data .
Qualifications
Must-Have
- 8+ years of dedicated professional experience in retail (e.g., merchandising, category management, retail operations, buying/planning) at a recognized, top-tier organization (e.g., Amazon, Walmart, Target, Nike, Costco, Home Depot, or equivalent).
- Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria — mandatory; please describe this experience in your application.
- Demonstrable career progression (e.g., Category Manager → Senior Manager → Director of Merchandising).
- Ability to engage reliably for at least 35 hours/week during weekdays.
- Verbal and written communication skills, problem-solving skills, and interpersonal skills.
Compensation & Legal
- W-2 employment with Cincinnatus LLC
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on us.fitly.work
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 8 days ago
Similar jobs that could be interesting for youBased on the Retail SME - AI Evaluation Expert in New York, NY vacancy
- Volga Partners is seeking experienced finance, audit, insurance, compliance, and legal professionals to join our on-call AI Evaluation Specialists network. This remote, project-based role involves evaluating AI outputs, applying critical thinking, and providing high-quality...SuggestedRemote jobFlexible hours
- SME Careers is seeking a Tax & Audit Subject Matter Expert to review AI-generated tax and audit responses. You will create expert accounting content, ensuring accuracy... .... This remote position offers flexibility to evaluate AI outputs across time zones. #J-18808-Ljbffr...SuggestedRemote work
- ...A leading AI research firm is seeking Expert Prompt Curators to design challenging prompts for evaluating advanced AI models. The role requires advanced knowledge in diverse fields and offers flexible hours, remote work, and a competitive hourly wage. Ideal candidates...SuggestedHourly payTemporary workRemote workFlexible hours
CloudDevs
New York, NY1 day ago - SME Careers is seeking a General Nurse Subject Matter Expert for a remote contractor role to review and create healthcare content... .... Your expertise will enhance AI models' accuracy in clinical... ...model integrity through rigorous evaluation of responses. #J-18808-Ljbffr SME...SuggestedRemote jobFor contractors
- A leading research services provider is seeking a Chemistry Expert (PhD) to evaluate complex chemistry problems and review AI-generated outputs for accuracy. This remote, hourly contract role requires deep subject-matter expertise and excellent communication skills. Candidates...SuggestedRemote jobHourly payContract workFlexible hours
$8 - $65 per hour
Prolific is hiring Mental Health Professionals in New York to train and evaluate AI models. As a Domain Expert, you will be responsible for reviewing AI-generated responses, completing tasks related to psychology, and improving AI models based on your expertise. Pay rates...Remote jobHourly payWork from homeFlexible hours$30 per hour
Prolific is seeking fluent Hindi speakers to join their Expert Network, helping to train and evaluate AI models with real legal expertise. Responsibilities include analyzing and writing tasks in Hindi, judging AI’s performance, and aiding in the improvement of AI models...Remote jobHourly payFlexible hours- Prolific in New York, NY, is seeking Chemistry Experts and Chemical Engineers to join its Expert Network. In this role, you will help train and evaluate AI models using your chemical expertise. Duties include evaluating AI-generated responses for accuracy and validating...Remote jobWork from homeFlexible hours
- Rise Data Labs is seeking advanced Mathematics and Statistics experts to support the training and evaluation of state-of-the-art AI systems. We need subject-matter experts who can apply deep quantitative knowledge to AI evaluation problems, assess AI-generated reasoning...Remote jobContract workImmediate startFlexible hours
- A tech company focusing on AI research is looking for experienced Krita users for a flexible, project-based contract opportunity. This role allows you to earn while evaluating AI-generated content related to digital painting and concept art. Candidates should have at least...Remote jobContract workFlexible hours
$60 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance...Contract workSummer workRemote workMercor
New York, NY10 days ago$80 - $150 per hour
Prolific in New York, NY, is seeking Medical Doctors to join our Expert Network for training and evaluating AI models. As a Domain Expert participant, you will review and rate AI-generated clinical responses, providing crucial feedback for AI development. You must hold...Remote jobHourly payWork from homeFlexible hours$30 per hour
About Prolific Prolific is not just another player in the AI space - we are building the biggest pool of quality human... ...Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image generation...Remote jobWork from homeFlexible hours- A leading AI Data Services company is seeking a bilingual content evaluator to review AI-generated responses and create training content. The ideal candidate will have... ...offers flexible hours and requires at least 3 years of relevant experience. #J-18808-Ljbffr SME CareersRemote jobFlexible hours
$30 per hour
Prolific is looking for Fluent Urdu Speakers to join our Expert Network to help train AI models using real legal expertise. The role involves completing tasks that require one hour of uninterrupted work, with pay rates of up to $30/hr. Candidates must possess advanced Urdu...Remote jobWork from homeFlexible hours- Mercor, in partnership with Crossing Hurdles, is looking for a Multimedia Expert to work remotely on project-based tasks. The role involves reviewing and editing multimedia outputs, creating assets for AI model training, and providing feedback to ensure quality standards....Remote jobPart time10 hours per week
- LILT (Production) is seeking a Subject Matter Expert to contribute to an AI benchmarking project in the healthcare sector. You will design realistic scenarios reflecting patient care and hospital operations, ensuring context and cultural appropriateness. Ideal candidates...For contractorsFlexible hours
- Handshake AI is seeking experienced CAD professionals with 2+ years of hands-on experience using SolidWorks and related CAD software, and professional proficiency in Mandarin Chinese, to support AI research through flexible, hourly contract work. This ongoing, project-...Remote jobHourly payContract workFlexible hours
- ...Remote Mathematics Expert (AI/LLM) - 34877 Remote Mathematics Expert (AI/LLM) - 34877 1 week ago Be among the first 25 applicants Turing... ...Collaborate with LLM researchers to align problem types with evaluation goals, particularly in areas where models commonly struggle (e...Hourly payContract workPart timeFor contractorsFreelanceInternshipRemote workWork from homeWorldwideAfternoon shift
Turing Inc
New York, NY1 day ago - ...Investment Banking AI Subject-Matter Expert is a remote review track for evaluating AI outputs across banking workflows. Reviewers grade calculations, narrative reasoning, and policy adherence; flag compliance and reconciliation issues; and document the correct treatment...Hourly payFor contractorsWork experience placementRemote work10 hours per week
AuraOne
New York, NY1 day ago $100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... .... Position: Cybersecurity Labeling Expert Type: Contract Compensation... ...ransomware , worms , and exploits . Evaluate POC exploit development to determine...Hourly payWeekly payFull timeContract workFor contractorsSummer workRemote workMercor
New York, NY12 hours ago$130 - $160 per hour
Mercor connects elite accounting talent with AI research labs. The Accounting Expert contract is remote, offering $130-$160/hour for candidates with deep US GAAP expertise and strong close-process skills. Responsibilities include designing grading criteria for accounting...Remote jobContract work$65 - $70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...and Jack Dorsey . Position: Biology Expert (PhD) — AI Safety Type: Contract... ...topics to enhance model understanding. Evaluate and annotate model responses for scientific...Contract workSummer workImmediate startRemote workMercor
New York, NYa month ago$80 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Help develop and refine clinician-informed evaluation rubrics for AI chatbot responses. Review... ...of AI-generated conversations and provide expert judgment to calibrate automated...Contract workTraineeshipSummer workRemote workMercor
New York, NY6 days ago$55 - $80 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Position: Automated CS Planning PhDs & Domain Experts — Zion (Arches) Type: Contract... ...creating complex problems in your domain. Evaluate and improve AI model performance through...Contract workSummer workRemote workMercor
New York, NYa month ago$150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...independently and asynchronously to meet deadlines while improving AI evaluation processes. Qualifications Must-Have US-based disease-...Hourly payContract workFor contractorsSummer workRemote workMercor
New York, NY12 days ago$60 - $75 per hour
...A leading information services firm is seeking a Remote Archivist to design questions and evaluate AI responses related to archival management. This position requires strong experience as an archivist and the ability to work independently in a fully remote environment....Hourly payRemote workCrossing Hurdles
New York, NY1 day ago$70 - $84 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...jailbreaks, unsafe behaviors, hallucinations, and policy failures. Evaluate model robustness across misinformation, cyber, biosecurity,...Contract workSummer workRemote workMercor
New York, NY6 days ago- ...hiring PhD‑level biologists to help make advanced AI models safer. You'll apply your scientific expertise to evaluate and strengthen how these models handle... ...train you on the workflow. Responsibilities Write expert‑level prompts across specialized life‑science topics...Part timeImmediate start
$50 per hour
Military Operations & IHL Expert.AI data lab for training frontier models and evaluating AI agents.Experts contribute their diverse subject matter knowledge across... ...believe t... Show more Full-time Quick Apply Kia Retailers are offering skilled and experienced military...Remote jobHourly payFull timeContract workPart timeFor contractorsApprenticeshipLocal areaWork from homeDay shift
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Retail SME - AI Evaluation Expert. Be the first to apply!
Related searches
- technology expert New York, NY
- subject matter expert New York, NY
- fulfillment expert New York, NY
- guest service support expert New York, NY
- sql expert New York, NY
- shift lead - retail New York, NY
- security officer retail store New York, NY
- retail finance New York, NY
- luxury retail part time New York, NY
- sports retail New York, NY