AI Evaluators: Assessing a Shopping Assistant

$30 per hour

Temporary

Terac

What You'll Do

Review real user interaction traces with an AI shopping assistant Identify logical failures, inaccuracies, or poor recommendations in the text Create structured rubrics and verifiers to judge response quality Commit to 20+ hours per week of evaluation work on our internal platform

Who Should Apply
Experience in data evaluation, quality assurance, or AI training Strong analytical skills with the ability to spot subtle errors in text Familiarity with e-commerce search and digital shopping experiences Ability to commit to a sustained workload of 20+ hours per week

Contract & Payment Terms
You will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities can be extended, shortened, or concluded early depending on needs and performance. Your participation will not involve access to confidential or proprietary information from any employer, client, or institution. Payments are processed weekly based on services rendered. We are unable to support H1-B or STEM OPT candidates at this time.

Pay: $30

Apply

Vacancy posted 19 days ago

Similar jobs that could be interesting for youBased on the AI Evaluators: Assessing a Shopping Assistant in San Francisco, CA vacancy

Health AI Conversation Evaluator
$20 - $160 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $... ...want to hear instead of what they need. Assess the tone and outcome of conversations. Determine...
Suggested
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
3 days ago
Analytical Evaluator - AI Feedback
$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Location: Remote Role Responsibilities Evaluate AI-generated responses to strengthen... ...and honest judgment, including critical assessments when warranted. Work independently and...
Suggested
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
15 days ago
Remote Healthcare AI Quality Evaluator
...Obsidian is hiring expert Evaluators in Healthcare operations for a remote, hourly engagement. You will review AI-generated work products for accuracy, rigor, and domain quality, leveraging your extensive subject-matter expertise in the field. The ideal candidate has...
Suggested
Hourly pay
Work at office
Remote work
Obsidian
San Francisco, CA
22 hours ago
Frontend Code Evaluator - Expert - AI Trainer
$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...from those that merely look correct. Evaluate responsive behavior and semantic quality,... ...evaluation work. Code review or technical assessment background. Pixel-perfect design-to-code...
Suggested
Contract work
Summer work
Local area
Remote work
Mercor
San Francisco, CA
8 days ago
PhD in Quantitative Finance, Remote AI Research Evaluator
$50 - $150 per hour
...hiring PhD-level quantitative finance experts to help train and evaluate advanced AI models on financial and quantitative reasoning. This is... ...You may contribute your expertise by: Assessing the factuality and relevance of quantitative and finance-related...
Suggested
Contract work
Temporary work
Remote work
Flexible hours
Braintrust
San Francisco, CA
19 days ago
Materials Science PhD - AI Evaluator
$70 per hour
...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers...
Contract work
Summer work
Immediate start
Remote work
Mercor
San Francisco, CA
27 days ago
Compliance AI Evaluator - Finance
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Compliance / regulatory response with financial-services AI Evaluator Type: Contract Compensation: $80–$120/hour...
Contract work
Summer work
Work at office
Remote work
Mercor
San Francisco, CA
28 days ago
AI Safety Practitioner - Expert Evaluator
$60 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance...
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
14 days ago
Data Analysis Expert - Evaluator
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $80–$120/hour...
Contract work
Summer work
Work at office
Remote work
Mercor
San Francisco, CA
13 days ago
Primary Care Physician - AI Evaluator
$170 - $190 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Primary Care Physician (MD) - Outpatient Clinical Documentation & AI Evaluation Type: Contract Compensation: $170–$190/hour...
Contract work
Work experience placement
Summer work
Remote work
Mercor
San Francisco, CA
22 days ago
Clinical Expert - AI System Evaluator
$150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...discrete, checkable criteria for consistent model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and...
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
14 days ago
CNC Machining Expert - Evaluator
$120 - $175 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...on CNC machining experience. ~ Strong production judgment to assess safe operations on real machines. ~ Written English fluency for...
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
11 days ago
Software Engineering Evaluator - Expert
$100 - $150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...00–$150/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts for usability in a software engineering...
Contract work
Summer work
Work at office
Remote work
Mercor
San Francisco, CA
a month ago
DevOps Engineer - AI Model Evaluator
$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks... ...outputs from multiple frontier models to assess their strengths and weaknesses. Apply...
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
8 days ago
AI Research Evaluator (STEM PhD/MS)
$30 - $90 per hour
...reviewers with strong STEM backgrounds to support quality control on AI reasoning datasets. This role requires professional fluency in... ...Attention to detail Nice to Have Experience with AI/ML evaluation or data annotation Background in AI, NLP, or related...
Contract work
Remote work
10 hours per week
Flexible hours
Braintrust
San Francisco, CA
19 days ago
AI Safety and Policy Evaluator
...Evaluate AI model safety by designing adversarial test prompts and classifications. About Turing: Based in San Francisco, California, Turing is the world's leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced...
Part time
For contractors
Turing
San Francisco, CA
13 days ago
JSON Evaluator - Fully Remote | Upto $175/hr
$175 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...0 hours/week Role Responsibilities Evaluate JSON-formatted AI model outputs against... ...descriptions and rubrics. Parse, read, and assess JSON-based outputs, grading them against...
Remote job
Contract work
Summer work
Start working today
Mercor
San Francisco, CA
11 days ago
Music Evaluator - Fully Remote | Upto $19/hr
$19 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...0 hours/week Role Responsibilities Evaluate AI-generated music across diverse genres... ...creativity, prompt adherence, and originality. Assess naturalness of lyrics, focusing on word...
Remote job
Contract work
Summer work
Immediate start
Mercor
San Francisco, CA
29 days ago
Machine Learning Evaluator - Fully Remote | Upto $90/hr
$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Location: Remote Role Responsibilities Evaluate the quality, correctness, and... ...tasks for training and evaluating models. Assess experiment design, model-selection reasoning...
Remote job
Contract work
Summer work
Mercor
San Francisco, CA
15 days ago
B2B Sales Expert - Evaluator
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...asynchronously to meet deadlines while improving evaluation processes. Qualifications Must-... ...may be asked to complete a brief assessment or submit additional information. Resources...
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
22 days ago
SWE Task Evaluator - Fully Remote | Upto $90/hr
$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Location: Remote Role Responsibilities Evaluate the quality, correctness, and... ...software-engineering benchmark tasks . Assess repository-level tasks, reference patches...
Remote job
Contract work
Summer work
Mercor
San Francisco, CA
15 days ago
Gmail & Google Calendar AI Assistant Evaluator
$100 per hour
...Required Skills Gmail Google Calendar AI Usage Scheduling Emails Google... ...lab for training frontier models and evaluating AI agents. Experts contribute their diverse... ...Role Title: Gmail & Google Calendar AI Assistant Evaluator Role Type: Contractor, Remote...
Part time
For contractors
Remote work
micro1
San Francisco, CA
19 days ago
AI Agent Evaluator
$25 - $45 per hour
...Required Skills ChatGPT Muse AI agents Attention to Detail Following... ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments... ..., returns, email follow-ups, bookings, shopping, groceries, travel, family logistics,...
Part time
For contractors
Work at office
Weekend work
micro1
San Francisco, CA
12 days ago
Hindi Music Specialist - Evaluator
$14 - $42 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...+ hours/week Role Responsibilities Evaluate AI-generated music across various genres.... ...creativity, prompt adherence, and originality. Assess naturalness of lyrics, including word...
Contract work
Summer work
Immediate start
Remote work
Mercor
San Francisco, CA
11 days ago
Grocery Clerk
$20.5 per hour
...operations Cart Attendant - Retrieve shopping carts from the parking lot and return... ...cleaning, and food production Baggers - Assist with bagging and other customer service... ...fast-changing world of work by sourcing, assessing, developing, and managing the talent...
Hourly pay
Temporary work
Shift work
3 days per week
Manpower
San Francisco, CA
1 day ago
Sales Engineering Expert - Evaluator
$100 - $150 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...objections, supporting proofs of concept, and evaluating product fit and technical risk. ~... ...applicants may be asked to complete a brief assessment or submit additional information....
Contract work
Summer work
Remote work
Mercor
San Francisco, CA
22 days ago
Primary Care Physician - AI Evaluator
$170 - $190 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Position: Primary Care Physician (MD) - Clinical Documentation & AI Evaluation Type: Contract Compensation: $170–$190/hour...
Contract work
Work experience placement
Summer work
Remote work
Mercor
San Francisco, CA
12 days ago
Nurse Practitioner - Clinical Evaluator
$70 - $110 per hour
...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our... ...Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated clinical assessments , treatment plans, and patient education materials for...
Hourly pay
Contract work
Summer work
Immediate start
Remote work
Mercor
San Francisco, CA
1 day ago
Cantonese Language Evaluator
$100 per hour
...Cantonese About micro1 micro1 is the leading AI data lab for training frontier models and evaluating AI agents. Experts contribute their diverse subject... ...accuracy of language technologies. Scope of Work Assess Cantonese text and speech data for linguistic...
Part time
For contractors
Remote work
micro1
San Francisco, CA
19 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Evaluators: Assessing a Shopping Assistant. Be the first to apply!