Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Context Elicitation & Conversation Evaluation Specialists - USA, Canada, Western Europe

Part-time

Turing

Review AI conversations to identify context gaps and improve clarity.

About Turing:
Turing is one of the world’s fastest-growing AI companies, accelerating the advancement and deployment of powerful AI systems.

Turing helps customers in two ways: Working with the world’s leading AI labs to advance frontier model capabilities in thinking, reasoning, coding, agentic behavior, multimodality, multilinguality, STEM, and frontier knowledge; and leveraging that work to build real-world AI systems that solve mission-critical priorities for companies.

Role overview :
We are seeking Subject Matter Experts (SMEs) with expertise in psychology, social work, counseling, qualitative research, HCI, communication, conflict resolution, coaching, or related advisory fields to support AI conversation evaluation research.

SMEs will review conversations between participants and AI models, assess whether sufficient context was gathered before recommendations were made, identify missed opportunities for clarification, and provide structured feedback to improve conversational quality.

Job Responsibilities:

  • Review AI conversations involving real-world interpersonal and everyday decision-making scenarios.
  • Identify missing context and recommend relevant follow-up questions.
  • Evaluate whether additional information would have changed the AI's recommendations.
  • Distinguish between critical ("load-bearing") and non-essential information.
  • Conduct qualitative reviews and document findings using structured evaluation guidelines.
  • Collaborate with research teams to ensure consistent, high-quality assessments..

Required Qualification:

  • Candidates should have at least 3–5 years of direct practicerelevant qualifications and professional experience in at least one of the following areas:

Licensed Social Work

  • MSW or equivalent graduate-level qualification in Social Work.
  • LCSW, LICSW, LMSW, or equivalent applicable social work license/credential.
  • Experience with client assessment, case review, interviewing, counseling, or related social-work practice.

Counseling, Therapy, or Psychology

  • MA/MS with LMFT, LPC, LCPC, or equivalent counseling/therapy license or credential; or
  • PhD/PsyD in Clinical, Counseling, Social, or Industrial-Organizational Psychology.
  • Experience with counseling, therapy, psychological assessment, interviewing, case review, or other relevant client-facing work.

Qualitative Research, HCI, or Communication

  • MA/PhD or equivalent advanced expertise in Human-Computer Interaction (HCI), Communication, Anthropology, Sociology, Qualitative Research, or a closely related discipline.
  • Experience conducting qualitative research, interviews, user research, conversation analysis, thematic analysis, qualitative coding, or similar research activities.
  • Relevant backgrounds may include Qualitative Research, HCI Research, UX/User Research, Communication Research, Anthropology, Sociology, or related disciplines.

Mediation, Coaching, Student Services, Organizational Consulting, or Related Advisory Fields

  • Relevant professional qualification, credential, or demonstrated subject-matter expertise.
  • Significant direct experience in advisory, client-facing, coaching, mediation, student advising/services, organizational consulting, or comparable work.
  • Experience gathering context, understanding individual or organizational needs, and providing informed recommendations or guidance.

    Must be based in the USA, Canada or Western Europe

Preferred experience:

  • Experience eliciting relevant context when clients initially provide incomplete information
  • Experience working across relationships, work, education, family, planning, wellbeing, and resource constraints, considering biological, psychological, social, spiritual/existential, financial, and practical factors
  • Qualitative coding or structured case review
  • Ability to distinguish between context that is load-bearing for the recommendation and that is merely descriptive/non-informative
  • Experience reviewing interviews, consultations, coaching sessions, or client interactions to identify missed opportunities for clarification or information
    gathering

    Project Details :
  • Project duration - 3 weeks (short term project)
  • Project type - Pay-per-task (PPT)
  • AHT - 75 minutes (approx.)
  • Engagement - Full time
  • Estimated project start date - 1st October (estimated)

Skills: Business Analysis

Vacancy posted 14 days ago
Similar jobs that could be interesting for youBased on the Context Elicitation & Conversation Evaluation Specialists - USA, Canada, Western Europe in San Francisco, CA vacancy
  • $20 per hour

     ...(AI Patient Access Specialist) Job Description The...  ...monitors AI-led conversations between healthcare...  ...skills to understand context from previous notes...  ...offices across the U.S., Canada, Asia Pacific and Europe, Aston Carter...  ...sourcing, screening, and evaluating candidates. AI... 
    Suggested
    Contract work
    Temporary work
    Internship
    Work at office
    Remote work

    Talentify

    San Francisco, CA
    4 days ago
  •  ...Job Application for Italian Audio Evaluations Specialist - Freelance AI Trainer Project at Meridial...  ...simulated interactions, audit conversational AI outputs, and generate clean, reliable...  ...about structured data within a support context. Representative Dataset Generation... 
    Suggested
    Hourly pay
    Part time
    For contractors
    Freelance
    Remote work

    Invisible Technologies

    San Francisco, CA
    2 days ago
  • $90 per hour

     ...DOM impact. ~ Professional written English for task justification. Preferred Prior RLHF , preference-labeling, model-evaluation, or structured code-review work. Pixel-perfect design-to-code experience. Accessibility expertise ( ARIA , semantic landmarks... 
    Suggested
    Contract work
    Summer work
    Local area
    Remote work

    Mercor

    San Francisco, CA
    23 days ago
  • $210.2k - $284.3k

     ...a Principal Data & Analytics Specialist Solutions Architect who will...  ...search.You assess the customer's context, design solutions using AWS...  ...experiences, including our Conversations on Race and Ethnicity (CORE)...  ...more about our benefits at .USA, CA, San Francisco - 210,200.... 
    Suggested
    Local area
    Worldwide
    Flexible hours

    AmazonWebServices

    San Francisco, CA
    1 day ago
  • $127k - $191.4k

     ...agents and automated workflows with context from over 750 integrations. Trusted by...  ...Workday feature releases, including evaluating new AI-powered capabilities for fit and...  ..., South Australia, Tasmania, Western Australia Canada: Alberta, Manitoba, Newfoundland, Northwest... 
    Suggested
    Work at office
    Local area

    PagerDuty

    San Francisco, CA
    4 days ago
  • $70 - $100 per hour

     ...Contract Compensation: $70–$100/hour Location: Remote Commitment: 10–20+ hours/week Role Responsibilities Evaluate AI-generated office artifacts across Excel , Word , and PowerPoint formats. Perform pairwise comparisons of candidate responses... 
    Contract work
    Summer work
    Work at office
    Immediate start
    Remote work

    Mercor

    San Francisco, CA
    a month ago
  • $11 - $30.65 per hour

     ...execute role-play–based evaluation scenarios that simulate realistic...  ...datasets used to assess conversational audio agents. Evaluate...  ...simulated customer support context Hindi proficiency...  .... Job title: Hindi Audio Specialist – AI Trainer Employment type... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Freelance
    Remote work

    Invisible Technologies

    San Francisco, CA
    2 days ago
  • $8 - $65 per hour

     ...Application for Korean-Japanese Bilingual Specialist - Freelance AI Trainer Project at...  ...communication across languages and contexts. That training data begins with you...  ...work with cutting-edge AI tools, evaluate translations and conversations between Korean and Japanese, and... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Freelance
    Remote work
    Worldwide

    Invisible Technologies

    San Francisco, CA
    2 days ago
  • $120k - $155k

     ...America, Latin America, and Europe. The rapid growth of our technology...  ...and Finance teams. Identify, evaluate, and prioritize scalable web...  ...both delightful and drives conversions. Growth Design: Based on 10+...  ...across the United States, Canada, Latin America, and Europe. This... 
    Work at office
    Local area
    Flexible hours

    Graphite

    San Francisco, CA
    3 days ago
  • $210.2k - $284.3k

     ...data workloads to AWS? Join the Worldwide Specialist Organization (WWSO) within the Media &...  ...and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon...  .... Learn more about our benefits at .USA, CA, San Francisco - 210,200.00 - 284,30... 
    Local area
    Worldwide
    Flexible hours

    AmazonWebServices

    San Francisco, CA
    3 days ago
  •  ...Application for Literature Specialist – Freelance AI Trainer Project...  ...are evolving from simple conversational tools into systems capable of...  ...style, symbolism, historical context, genre conventions, and close...  ...accuracy and coherence; evaluate the depth and validity of literary... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Freelance
    Remote work

    Invisible Technologies

    San Francisco, CA
    2 days ago
  •  ...area and offices in Michigan and soon in Europe. Maxterial is a mid-stage VC-backed innovative...  ...technology. Analytical Chemistry Specialist Responsibilities: The scale up support...  ...status with authorization to work in the USA. What makes you a technically perfect... 
    Immediate start
    Relocation package

    Maxterial, Inc.

    San Francisco, CA
    2 days ago
  • $150 per hour

     ...independently and asynchronously to enhance search quality and evaluation. Qualifications Must-Have Experience with production...  ...impact and improvements. Interview Process ~25 mins conversational interview. No coding or take-home tasks. ~ Follow-up 30 mins... 
    Remote job
    Contract work
    Summer work

    Mercor

    San Francisco, CA
    24 days ago
  • Travelwithsan is seeking a motivated Lead Generation Specialist to identify, engage, and develop relationships with potential...  ...to contribute to business growth from anywhere in the USA or LATAM. You will start conversations, qualify leads, and maintain a healthy prospect... 
    Remote job
    Flexible hours

    Travelwithsan

    San Francisco, CA
    5 days ago
  • Lead Generation Specialist Job Type: Remote Schedule: Flexible / [Part-Time or Full-Time] Location: Remote USA or LATAM About the Role We are seeking a motivated Lead Generation...  ...enjoys connecting with people, starting conversations, following up consistently, and... 
    Full time
    Part time
    Remote work
    Flexible hours

    Travelwithsan

    San Francisco, CA
    5 days ago
  •  ...Lead Generation Specialist We are seeking a proactive and results-driven Lead Generation...  ...other channels. Qualify leads by evaluating their needs, business requirements, interest...  ...and engage prospects in meaningful conversations. Identify decision-makers and relevant... 
    Remote work

    Simera

    San Francisco, CA
    4 days ago
  • $30 per hour

     ...content and trainer/QA work, evaluate output quality against...  ...translation fidelity, cultural context, terminology, and edge cases...  ...handling formal, neutral, and conversational Ukrainian, including audience...  ...Open to candidates in: Canada, Kazakhstan, Moldova, Poland... 
    Hourly pay
    Temporary work
    For contractors
    Remote work

    SuperAnnotate

    San Francisco, CA
    2 days ago
  •  ...preferences, needs, and concerns. •    Engage in meaningful conversations to gather detailed information about customers' sleep...  ...mattresses, providing expert guidance on proper posture, comfort evaluation, and support assessment. •    Stay up-to-date with current... 
    Sleeping nights
    Weekend work

    MANCINI'S SLEEPWORLD

    San Francisco, CA
    1 day ago
  • $20 per hour

     ...Job Application for Search Engine Evaluation Specialist - Freelance AI Trainer Project at Meridial Back to jobs Search Engine Evaluation...  ...This includes the ability to participate in professional conversations, explain nuanced concepts, and understand detailed... 
    Hourly pay
    Part time
    For contractors
    Freelance
    Remote work

    Invisible Technologies

    San Francisco, CA
    14 days ago
  •  ...! ABOUT THE ROLE We are looking for an AI Platform Context Lead to build the context and knowledge layer behind an enterprise...  ..., and AI assistants. You will improve answer quality through evaluation, governance, and reusable self-service patterns. WHAT YOU... 
    Work at office
    Work from home
    Visa sponsorship
    Work visa

    AgileEngine

    San Francisco, CA
    3 days ago
  •  ...seeking a Lakebase Associate Director Specialists to help customers modernize their operational...  ....The interview process is designed to evaluate candidates across all four of these...  ...whiteboard architectures, lead C-level conversations, and build trust with senior decision makers... 
    Worldwide

    DataBricks

    San Francisco, CA
    3 days ago
  • $100k - $150k

     ...insightsConnect leadership insight to business context, company performance, value creation,...  ...are signing up for and how you will be evaluated.Project managementManages overall...  ...debriefs, assessments, workshops, and client conversations without being promptedCoordinates team... 
    Full time
    Temporary work
    Apprenticeship
    Casual work
    Work at office
    Local area
    Immediate start
    Flexible hours
    Shift work
    2 days per week
    3 days per week

    Spencer Stuart

    San Francisco, CA
    9 hours ago
  • $89.59k - $197.46k

     ...own future.KPMG is currently seeking a Senior Associate, Credit Specialist to join our Audit practice.Responsibilities:Execute the day-to...  ...models to credit loss accounting concepts, including evaluation of model validation design and application Ability to meet challenging... 
    H1b
    Local area

    KPMG

    San Francisco, CA
    3 days ago
  •  ...proactive AI assistant that understands context and delivers help automatically. Founded...  ...clearly and with empathy in writing and conversation, and builds trusting relationships with...  ...also abide by the Employment Equity Act (Canada).Compensation Range: $94K - $130KLocationHub... 
    Contract work
    Worldwide
    Home office
    Flexible hours
    2 days per week

    Superhuman

    San Francisco, CA
    2 days ago
  • $80 - $120 per hour

     ...Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey . Position: IP / trademark / copyright law Evaluator Type: Contract Compensation: $80–$120/hour Location: Remote Role Responsibilities Evaluate AI-generated artifacts... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    24 days ago
  • $95k - $105k

     ...communications while recognizing which conversations need senior relationship ownership — and...  ...Intelligence Help the Growth Leader evaluate where 2B Living should invest time and...  ...business enters onboarding with complete context: commitments made, client priorities,... 
    Local area

    2B LIVING INC

    San Francisco, CA
    1 day ago
  •  ...productivity, and performance — and use that context, together with rigorous analysis, to...  ...ambiguous problems, test new ideas, evaluate opportunities and trade-offs, and carry...  ...inform our Recruitment team during your conversation with them.To learn more about our culture... 
    Work at office
    Local area

    Atlassian

    San Francisco, CA
    10 hours ago
  • $151k - $204.3k

     ...Bedrock 2) Agentic AI including Agentic platforms, tool usage, evaluations, governance and cost management along with use of Agentic SDKs...  ...Austin, Dallas, Atlanta, Arlington. Travel up to 50% across the USA may be possible.Key job responsibilitiesWorking with customers’... 
    Local area
    Worldwide
    Flexible hours

    AmazonWebServices

    San Francisco, CA
    3 days ago
  •  ...are currently looking for a Paid Search Specialist to join our growing media team. This...  ...trackers, and optimization records. Support conversion tracking, UTM governance, and tagging...  ...skills. All employment, promotion, and evaluation decisions are based on qualifications,... 
    Work at office

    Prophet

    San Francisco, CA
    2 days ago
  • $120k - $130k

     ...: by sharing knowledge, experience, and context that can't be captured in code alone. Today...  ...to medical professionals—to train and evaluate cutting‑edge models. Your work will...  ...sourcing tools. Comfortable driving high conversion rates and managing funnels effectively.... 
    Work at office
    Relocation package

    Mercor Inc

    San Francisco, CA
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Context Elicitation & Conversation Evaluation Specialists - USA, Canada, Western Europe. Be the first to apply!