AI Evaluators: Assessing a Shopping Assistant
$30 per hourTerac
What You'll Do
Review real user interaction traces with an AI shopping assistant Identify logical failures, inaccuracies, or poor recommendations in the text Create structured rubrics and verifiers to judge response quality Commit to 20+ hours per week of evaluation work on our internal platform
Who Should Apply
Experience in data evaluation, quality assurance, or AI training Strong analytical skills with the ability to spot subtle errors in text Familiarity with e-commerce search and digital shopping experiences Ability to commit to a sustained workload of 20+ hours per week
Contract & Payment Terms
You will be engaged as an independent contractor. This is a fully remote opportunity that can be completed on your own schedule. Opportunities can be extended, shortened, or concluded early depending on needs and performance. Your participation will not involve access to confidential or proprietary information from any employer, client, or institution. Payments are processed weekly based on services rendered. We are unable to support H1-B or STEM OPT candidates at this time.
Pay: $30
- ...Join a 16-week research study evaluating AI assistants from the perspective of everyday users. We’re looking for people who already use Gemini, Instinct, or Muse and can assess prompts and responses in English. A part-time commitment of 20 hours per week is available...SuggestedFull timePart timeImmediate start
- Mercor is seeking a Procurement / Vendor Management Evaluator for a remote contract position. The role focuses on assessing AI-generated artifacts, identifying errors, and providing structured feedback to improve AI model outputs. You will collaborate with AI research...SuggestedRemote jobContract work
$90 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...from those that merely look correct. Evaluate responsive behavior and semantic quality,... ...evaluation work. Code review or technical assessment background. Pixel-perfect design-to-code...SuggestedContract workSummer workLocal areaRemote work$85 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...frontier AI coding agents to complete and evaluate complex infrastructure engineering tasks... ...outputs from multiple frontier models to assess their strengths and weaknesses. Apply...SuggestedContract workSummer workRemote work- ...Francisco, Turing is the world’s leading research accelerator for frontier AI labs and a trusted partner for global enterprises deploying advanced AI systems. As a Software Engineering evaluator, you will curate code examples, provide precise solutions, and refine AI-generated...SuggestedFor contractors
$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Dorsey . Position: Incident management / reliability / SRE Evaluator Type: Contract Compensation: $80–$120/hour Location...Contract workSummer workWork at officeRemote work$20 - $160 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Annotator — Health AI Conversation Quality Evaluation Type: Contract Compensation: $... ...want to hear instead of what they need. Assess the tone and outcome of conversations. Determine...Contract workSummer workRemote work$70 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...Location: Remote Role Responsibilities Evaluate AI-generated responses to strengthen... ...and honest judgment, including critical assessments when warranted. Work independently and...Contract workSummer workRemote work$20 per hour
...Experience: 1 - 3 years Help improve leading AI models through research, reasoning and educational evaluation. This remote opportunity is for students and... ...least 10 educational conversations. An additional assessment will be shared after initial review. Company:...Contract workImmediate startRemote work$50 - $150 per hour
...hiring PhD-level quantitative finance experts to help train and evaluate advanced AI models on financial and quantitative reasoning. This is... ...You may contribute your expertise by: Assessing the factuality and relevance of quantitative and finance-related...Contract workTemporary workRemote workFlexible hours$10 - $30 per hour
...feedback digital delivery platforms ai-driven tools familiarity independent... ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments... ...is what matters. Scope of Work: Assess and provide feedback on AI-generated...Part timeFor contractorsRemote workShift work$70 - $110 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...weeks Commitment: 20+ hours/week Role Responsibilities Evaluate AI-generated operational plans , staff schedules, and performance...Hourly payContract workSummer workImmediate startRemote work$290k - $365k
...create reliable, interpretable, and steerable AI systems. We want AI to be safe and... ...role Anthropic gives independent external evaluators access to our models, our people, and our safety practices so that they can assess risks, processes, and policies. We're hiring...Contract workWork at officeLocal areaVisa sponsorshipFlexible hours- ...Luxury Brand Evaluator CXG is a global Customer Experience consulting firm specializing exclusively in serving premium and luxury brands... ..., you will step into the world of luxury to discreetly assess customer experiences, providing critical feedback that helps brands...WorldwideFlexible hours
- Mercor is seeking an AI Safety Practitioner for a remote contract role. You will evaluate AI-generated responses for safety, accuracy, and policy compliance, covering sensitive domains. You will collaborate with AI researchers and safety teams to refine evaluation rubrics...Remote jobContract workFlexible hours
- ...Become a Luxury Automotive Experience Evaluator As a Luxury Automotive Experience Evaluator... ...eye for luxury Observe every detail - Assess the ambiance, level of service, digital... ...observing and noting various aspects of your shopping experience Own a car and has a driving...FreelanceFlexible hours
- A leading luxury brand evaluator is seeking a Luxury Automotive Experience Evaluator in San Francisco. This freelance position offers flexible working hours and invites automotive enthusiasts to assess premium showrooms based on their refined experiences. You will engage...FreelanceFlexible hours
- Cypress HCM is seeking an LLM Evaluation Specialist for a part-time, contract role in the San Francisco... ...area. You will run evaluation passes on AI-generated charts, headlines, captions, and chats, using a defined rubric to assess accuracy and clarity. You’ll log findings...Remote jobContract workPart time
- Mercor is seeking a Biology / environmental science Evaluator for a remote contract role. You will assess AI-generated artifacts using domain-specific rubrics, identify errors, and provide structured feedback to improve content quality. Candidates should have 5+ years...Remote jobContract workWork at office
- ...About the role AI assistants are becoming part of everyday life, helping people with everything from writing emails and planning trips... ...AI responses and select your preferred experience. Evaluate conversational quality, helpfulness, clarity, tone, and overall...Hourly payTemporary workPart timeRemote work
$70 per hour
...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers...Contract workSummer workImmediate startRemote work$20.5 per hour
...grocery operations Cart Attendant - Retrieve shopping carts from the parking lot and return... ...cleaning, and food productionBaggers - Assist with bagging and other customer service... ...fast-changing world of work by sourcing, assessing, developing, and managing the talent...Hourly payShift work3 days per week$180 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...5 hours/week Role Responsibilities Evaluate AI-generated legal responses to enhance... ...law students and live-client clinics to assess AI-generated legal analyses . Work...Contract workSummer workRemote work$150 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...discrete, checkable criteria for consistent model response grading. Evaluate clinical dialogues for accuracy, safety, completeness, and...Contract workSummer workRemote work$25 - $45 per hour
...Required Skills ChatGPT Muse AI agents Attention to Detail Following... ...foundational models and agents. We provide frontier evaluations and reinforcement learning environments... ..., returns, email follow-ups, bookings, shopping, groceries, travel, family logistics,...Part timeFor contractorsWork at officeWeekend work$80 - $120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...Jack Dorsey . Position: Data analysis / quantitative readouts Evaluator Type: Contract Compensation: $80–$120/hour...Contract workSummer workWork at officeRemote work$30 per hour
...creative and technical talent with leading AI research labs. Headquartered in San Francisco... ...user would type. Collaborate with the AI assistant on tasks, guiding it, answering questions, and correcting errors. Evaluate the experience and deliverables, providing concise...Hourly payWeekly payContract workFor contractorsSummer workRemote work$60 - $70 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ...$60–$70/hour Location: Remote Role Responsibilities Evaluate AI-generated responses for safety, factual accuracy, policy compliance...Contract workSummer workRemote work$120 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., and Jack Dorsey . Position: Procurement / vendor management Evaluator Type: Contract Compensation: $80-$120/hour Location: Remote Role...Remote jobContract workSummer workWork at office- Mercor is seeking a Punjabi-language Music & Lyrics Expert for a remote contract role. You will evaluate AI-generated music and lyrics, rate quality, and compare lyrics to published works while maintaining high standards. The position offers up to 6 months with a 20-hour...Remote jobContract workPart timeImmediate start
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Evaluators: Assessing a Shopping Assistant. Be the first to apply!
- evaluator San Francisco, CA
- work from home web search evaluator San Francisco, CA
- ai evaluator San Francisco, CA
- education evaluator San Francisco, CA
- quality evaluator San Francisco, CA
- social media evaluator San Francisco, CA
- body shop assistant San Francisco, CA
- coffee shop attendant San Francisco, CA
- shop helper San Francisco, CA
- store associate San Francisco, CA






