Retail SME - AI Evaluation Expert
$60 - $80 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: Retail Specialist
Type: Contract
Compensation: $60–$80/hour
Location: Remote
Role Responsibilities
- Guide research and engineering teams to close knowledge gaps in retail merchandising , category management , and operations reasoning .
- Design challenging, domain-relevant retail tasks and write accurate, well-reasoned solutions grounded in real retail practice.
- Evaluate AI model outputs against structured rubrics and provide clear, written feedback on correctness, judgment, and reasoning quality.
- Develop and refine evaluation guidelines and scoring rubrics specific to retail tasks.
- Collaborate with other subject matter experts to ensure consistency and accuracy in training data .
Qualifications
Must-Have
- 8+ years of dedicated professional experience in retail (e.g., merchandising, category management, retail operations, buying/planning) at a recognized, top-tier organization (e.g., Amazon, Walmart, Target, Nike, Costco, Home Depot, or equivalent).
- Prior hands-on experience evaluating LLM/AI model outputs against rubrics or structured scoring criteria.
- Demonstrable career progression (e.g., Category Manager → Senior Manager → Director of Merchandising).
- Ability to engage reliably for at least 35 hours/week during weekdays.
- Verbal and written communication skills, problem-solving skills, and interpersonal skills.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on us.fitly.work
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 2 days ago
Similar jobs that could be interesting for youBased on the Retail SME - AI Evaluation Expert in New York, NY vacancy
- 1. Overview Join a leading AI lab's cutting-edge GenAI team and help build foundational... ...from the ground up. We’re seeking talented Retail subject-matter experts (SMEs) with deep domain expertise and hands-on experience evaluating AI model outputs against rubrics to bring...SuggestedContract workWeekday work
- Volga Partners is seeking experienced finance, audit, insurance, compliance, and legal professionals to join our on-call AI Evaluation Specialists network. This remote, project-based role involves evaluating AI outputs, applying critical thinking, and providing high-quality...SuggestedRemote jobFlexible hours
- SME Careers is seeking a Tax & Audit Subject Matter Expert to review AI-generated tax and audit responses. You will create expert accounting content, ensuring accuracy... .... This remote position offers flexibility to evaluate AI outputs across time zones. #J-18808-Ljbffr...SuggestedRemote work
- SME Careers is seeking a remote Data Scientist to contribute to AI training content and ensure model integrity. The role involves developing AI prompts, evaluating AI responses, and testing for model reliability. Ideal candidates will have a degree in a quantitative field...SuggestedRemote jobHourly payFor contractors
- HireArt is seeking a Godot Software Expert for a short-term AI evaluation project. You will design complex Godot workflows and record end-to-end tasks for quality evaluation. This contract position runs ~1.5 weeks and is remote from the United States, with flexible hours...SuggestedRemote jobContract workTemporary workPart time10 hours per weekFlexible hours
$30 per hour
Prolific is seeking fluent Hindi speakers to join their Expert Network, helping to train and evaluate AI models with real legal expertise. Responsibilities include analyzing and writing tasks in Hindi, judging AI’s performance, and aiding in the improvement of AI models...Remote jobHourly payFlexible hours- A tech company focusing on AI research is looking for experienced Krita users for a flexible, project-based contract opportunity. This role allows you to earn while evaluating AI-generated content related to digital painting and concept art. Candidates should have at least...Contract workRemote workFlexible hours
Handshake
New York, NY20 hours ago - SME Careers is seeking a Financial Analysis Subject Matter Expert (SME) to work remotely on an hourly basis. You will be responsible for reviewing AI-generated financial analyses and providing expert content while ensuring accuracy and clarity in responses. The ideal candidate...Remote jobHourly pay
- Hebrew Bilingual Expert — AI Evaluation & Annotation (Remote) is a remote evaluation track for reviewing hebrew generalist evaluation prompts and responses against AuraOne's quality rubric. Reviewers compare paired outputs, label edge cases, and write the kind of structured...Remote jobHourly payFor contractors10 hours per week
- HireArt is seeking an experienced Audacity Software Expert for a short-term AI evaluation project. You will design complex Audacity tasks, develop evaluation rubrics, and record workflows from start to finish. This contract role focuses on high-quality, professional outputs...Remote jobContract workTemporary workFlexible hours
$73 per hour
A leading AI innovation firm is seeking individuals for a flexible, project-based role focusing on creating complex tasks for evaluating AI performance. Ideal candidates should possess a postgraduate degree and relevant industry experience, especially in economics. This...Remote workFlexible hours- A leading research services provider is seeking a Chemistry Expert (PhD) to evaluate complex chemistry problems and review AI-generated outputs for accuracy. This remote, hourly contract role requires deep subject-matter expertise and excellent communication skills. Candidates...Remote jobHourly payContract workFlexible hours
- Mercor is seeking a Music Audio Expert - German for a remote, project-based engagement. You will evaluate AI output lyrics and voice generation, score training data quality, and write music in the domain language listed in the title. Applicants should have 3+ years as...Remote job10 hours per week
- A leading AI research firm is seeking Expert Prompt Curators to design challenging prompts for evaluating advanced AI models. The role requires advanced knowledge in diverse fields and offers flexible hours, remote work, and a competitive hourly wage. Ideal candidates will...Remote jobHourly payTemporary workFlexible hours
- Expert Prompt Curators for Advanced AI Evaluation Dataset This description is a summary of our understanding of the job description. Click on ‘Apply’ button to find out more. Role Description Mercor is collaborating with a leading AI research lab to develop a next-generation...Hourly payContract workTemporary workFor contractorsRemote work
- Mercor is seeking experienced musicians to evaluate generative music AI models, collaborating with a leading AI lab. You will assess AI-generated music and lyrics in Telugu and English, applying detailed quality standards across genres. Responsibilities include comparing...Part timeImmediate start
- Rise Data Labs is seeking advanced Mathematics and Statistics experts to support the training and evaluation of state-of-the-art AI systems. We need subject-matter experts who can apply deep quantitative knowledge to AI evaluation problems, assess AI-generated reasoning...Remote jobContract workImmediate startFlexible hours
$80 - $150 per hour
Prolific in New York, NY, is seeking Medical Doctors to join our Expert Network for training and evaluating AI models. As a Domain Expert participant, you will review and rate AI-generated clinical responses, providing crucial feedback for AI development. You must hold...Remote jobHourly payWork from homeFlexible hours- ...Physics Specialist to contribute deep scientific expertise to AI model evaluation. You will craft and assess challenging physics problems,... ...electromagnetism, use adversarial prompting to surface errors, and provide expert critique of AI responses while working with project #J-1880...Remote job
- Mercor is hiring experienced music producers and audio engineers to evaluate generative music AI models, in partnership with a leading AI lab. You will assess AI-generated music across genres and rate it against detailed quality standards, working in Vietnamese and English...Immediate startFlexible hours
$30 per hour
About Prolific Prolific is not just another player in the AI space - we are building the biggest pool of quality human... ...Graphic and Visual Designers to act as Domain Experts for a high-level AI evaluation project. AI models are evolving beyond simple image generation...Remote jobWork from homeFlexible hours- Turing Global India is seeking a Politics Domain Expert to evaluate and improve Large Language Models. You will design challenging prompts across political science, governance, elections, public policy and international relations, and assess factual accuracy and reasoning...For contractors
- Mercor, in partnership with Crossing Hurdles, is looking for a Multimedia Expert to work remotely on project-based tasks. The role involves reviewing and editing multimedia outputs, creating assets for AI model training, and providing feedback to ensure quality standards....Remote jobPart time10 hours per week
$30 per hour
Prolific is looking for Fluent Urdu Speakers to join our Expert Network to help train AI models using real legal expertise. The role involves completing tasks that require one hour of uninterrupted work, with pay rates of up to $30/hr. Candidates must possess advanced Urdu...Remote jobWork from homeFlexible hours- A leading AI Data Services company is seeking a bilingual content evaluator to review AI-generated responses and create training content. The ideal candidate will have... ...offers flexible hours and requires at least 3 years of relevant experience. #J-18808-Ljbffr SME CareersRemote jobFlexible hours
- Cincinnatus LLC is hiring for a Marketing SME to support GenAI model evaluation and brand/growth tasks. The role centers on applying rigorous marketing judgment to AI training data and guiding cross-functional teams to improve model outputs. Ideal candidates have extensive...
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., and Jack Dorsey . Position: Software / AI / IT / data Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work$60 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance...Contract workSummer workRemote work- ...Investment Banking AI Subject-Matter Expert is a remote review track for evaluating AI outputs across banking workflows. Reviewers grade calculations, narrative reasoning, and policy adherence; flag compliance and reconciliation issues; and document the correct treatment...Hourly payFor contractorsWork experience placementRemote work10 hours per week
AuraOne
New York, NY1 day ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Retail SME - AI Evaluation Expert. Be the first to apply!
Related searches