Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Evaluators: Assessing a Shopping Assistant

$30 per hour
Temporary

Terac

What You'll Do

Review real user interaction traces with an AI shopping assistant Identify logical failures, inaccuracies, or poor recommendations in the text Create structured rubrics and verifiers to judge response quality Commit to 20+ hours per week of evaluation work on our internal platform

Who Should Apply
Experience in data evaluation, quality assurance, or AI training Strong analytical skills with the ability to spot subtle errors in text Familiarity with e-commerce search and digital shopping experiences Ability to commit to a sustained workload of 20+ hours per week

Contract & Payment Terms
You will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities can be extended, shortened, or concluded early depending on needs and performance. Your participation will not involve access to confidential or proprietary information from any employer, client, or institution. Payments are processed weekly based on services rendered. We are unable to support H1-B or STEM OPT candidates at this time.

Pay: $30

Vacancy posted a month ago
Similar jobs that could be interesting for youBased on the AI Evaluators: Assessing a Shopping Assistant in San Francisco, CA vacancy
  •  ...Join a 16-week research study evaluating AI assistants from the perspective of everyday users. We’re looking for people who already use Gemini, Instinct, or Muse and can assess prompts and responses in English. A part-time commitment of 20 hours per week is available... 
    Suggested
    Full time
    Part time
    Immediate start

    Braintrust

    San Francisco, CA
    1 day ago
  • Mercor is seeking a Procurement / Vendor Management Evaluator for a remote contract position. The role focuses on assessing AI-generated artifacts, identifying errors, and providing structured feedback to improve AI model outputs. You will collaborate with AI research... 
    Suggested
    Remote job
    Contract work

    Mercor

    San Francisco, CA
    5 days ago
  • $90 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...from those that merely look correct. Evaluate responsive behavior and semantic quality,...  ...evaluation work. Code review or technical assessment background. Pixel-perfect design-to-code... 
    Suggested
    Contract work
    Summer work
    Local area
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  • $85 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks...  ...outputs from multiple frontier models to assess their strengths and weaknesses. Apply... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    4 days ago
  •  ...Francisco, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. As a Software Engineering evaluator, you will curate code examples, provide precise solutions, and refine AI-generated... 
    Suggested
    For contractors

    Parailabs

    San Francisco, CA
    4 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Dorsey . Position: Incident management / reliability / SRE Evaluator Type: Contract Compensation: $80–$120/hour Location... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $20 - $160 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $...  ...want to hear instead of what they need. Assess the tone and outcome of conversations. Determine... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $70 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...Location: Remote Role Responsibilities Evaluate AI-generated responses to strengthen...  ...and honest judgment, including critical assessments when warranted. Work independently and... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    27 days ago
  • $20 per hour

     ...Experience: 1 - 3 years Help improve leading AI models through research, reasoning and educational evaluation. This remote opportunity is for students and...  ...least 10 educational conversations. An additional assessment will be shared after initial review. Company:... 
    Contract work
    Immediate start
    Remote work

    Braintrust

    San Francisco, CA
    21 days ago
  • $50 - $150 per hour

     ...hiring PhD-level quantitative finance experts to help train and evaluate advanced AI models on financial and quantitative reasoning. This is...  ...You may contribute your expertise by: Assessing the factuality and relevance of quantitative and finance-related... 
    Contract work
    Temporary work
    Remote work
    Flexible hours

    Braintrust

    San Francisco, CA
    a month ago
  • $10 - $30 per hour

     ...feedback digital delivery platforms ai-driven tools familiarity independent...  ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments...  ...is what matters. Scope of Work: Assess and provide feedback on AI-generated... 
    Part time
    For contractors
    Remote work
    Shift work

    micro1

    San Francisco, CA
    21 days ago
  • $70 - $110 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...weeks Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated operational plans , staff schedules, and performance... 
    Hourly pay
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    27 days ago
  • $290k - $365k

     ...create reliable, interpretable, and steerable AI systems. We want AI to be safe and...  ...role Anthropic gives independent external evaluators access to our models, our people, and our safety practices so that they can assess risks, processes, and policies. We're hiring... 
    Contract work
    Work at office
    Local area
    Visa sponsorship
    Flexible hours

    Anthropic

    San Francisco, CA
    1 day ago
  •  ...Luxury Brand Evaluator CXG is a global Customer Experience consulting firm specializing exclusively in serving premium and luxury brands...  ..., you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands... 
    Worldwide
    Flexible hours

    CXG

    San Francisco, CA
    4 days ago
  • Mercor is seeking an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance, covering sensitive domains. You will collaborate with AI researchers and safety teams to refine evaluation rubrics... 
    Remote job
    Contract work
    Flexible hours

    Mercor

    San Francisco, CA
    3 days ago
  •  ...Become a Luxury Automotive Experience Evaluator As a Luxury Automotive Experience Evaluator...  ...eye for luxury Observe every detail - Assess the ambiance, level of service, digital...  ...observing and noting various aspects of your shopping experience Own a car and has a driving... 
    Freelance
    Flexible hours

    CXG

    San Francisco, CA
    5 days ago
  • A leading luxury brand evaluator is seeking a Luxury Automotive Experience Evaluator in San Francisco. This freelance position offers flexible working hours and invites automotive enthusiasts to assess premium showrooms based on their refined experiences. You will engage... 
    Freelance
    Flexible hours

    CXG

    San Francisco, CA
    3 days ago
  • Cypress HCM is seeking an LLM Evaluation Specialist for a part-time, contract role in the San Francisco...  ...area. You will run evaluation passes on AI-generated charts, headlines, captions, and chats, using a defined rubric to assess accuracy and clarity. You’ll log findings... 
    Remote job
    Contract work
    Part time

    Cypress HCM

    San Francisco, CA
    4 days ago
  • Mercor is seeking a Biology / environmental science Evaluator for a remote contract role. You will assess AI-generated artifacts using domain-specific rubrics, identify errors, and provide structured feedback to improve content quality. Candidates should have 5+ years... 
    Remote job
    Contract work
    Work at office

    Mercor

    San Francisco, CA
    4 days ago
  •  ...About the role AI assistants are becoming part of everyday life, helping people with everything from writing emails and planning trips...  ...AI responses and select your preferred experience. Evaluate conversational quality, helpfulness, clarity, tone, and overall... 
    Hourly pay
    Temporary work
    Part time
    Remote work

    micro1

    San Francisco, CA
    4 days ago
  • $70 per hour

     ...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers... 
    Contract work
    Summer work
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    23 days ago
  • $20.5 per hour

     ...grocery operations Cart Attendant - Retrieve shopping carts from the parking lot and return...  ...cleaning, and food productionBaggers - Assist with bagging and other customer service...  ...fast-changing world of work by sourcing, assessing, developing, and managing the talent... 
    Hourly pay
    Shift work
    3 days per week

    Manpower

    San Francisco, CA
    1 day ago
  • $180 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...5 hours/week Role Responsibilities Evaluate AI-generated legal responses to enhance...  ...law students and live-client clinics to assess AI-generated legal analyses . Work... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  • $150 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...discrete, checkable criteria for consistent model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    10 days ago
  • $25 - $45 per hour

     ...Required Skills ChatGPT Muse AI agents Attention to Detail Following...  ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments...  ..., returns, email follow-ups, bookings, shopping, groceries, travel, family logistics,... 
    Part time
    For contractors
    Work at office
    Weekend work

    micro1

    San Francisco, CA
    13 days ago
  • $80 - $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Jack Dorsey . Position: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $80–$120/hour... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    9 days ago
  • $30 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San Francisco...  ...user would type. Collaborate with the AI assistant on tasks, guiding it, answering questions, and correcting errors. Evaluate the experience and deliverables, providing concise... 
    Hourly pay
    Weekly pay
    Contract work
    For contractors
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    18 days ago
  • $60 - $70 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  • $120 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ..., and Jack Dorsey . Position: Procurement / vendor management Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Role... 
    Remote job
    Contract work
    Summer work
    Work at office

    Mercor

    San Francisco, CA
    5 days ago
  • Mercor is seeking a Punjabi-language Music & Lyrics Expert for a remote contract role. You will evaluate AI-generated music and lyrics, rate quality, and compare lyrics to published works while maintaining high standards. The position offers up to 6 months with a 20-hour... 
    Remote job
    Contract work
    Part time
    Immediate start

    Mercor

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Evaluators: Assessing a Shopping Assistant. Be the first to apply!