Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI Conversation Evaluation Specialist | $32/hr Remote

$32 per hour

Crossing Hurdles

New York, NY
  • Remote job

Position: Text Situated Alignment Specialist Type: Contractor Assignment Compensation: $32/hour Location: Remote Commitment: Full-time (40 hours/week), for 3 weeks Role Responsibilities Review AI conversations involving real-world interpersonal and everyday decision-making scenarios. Identify missing context and recommend relevant follow-up questions. Evaluate whether additional information could have changed the AI s recommendations. Distinguish between critical, decision-relevant information and non-essential details. Conduct qualitative reviews and document findings using structured evaluation guidelines. Requirements 3+ years of experience in psychology, social work, counseling, qualitative research, HCI, communication, conflict resolution, coaching, student services, organizational consulting, or related advisory fields. Hands-on experience with client assessment, interviewing, counseling, case review, qualitative research, user research, conversation analysis, or similar work. Experience identifying relevant context and determining which information is important for informed recommendations. Experience with qualitative coding, structured case review, or reviewing client interactions is preferred. #J-18808-Ljbffr Crossing Hurdles

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Conversation Evaluation Specialist | $32/hr Remote in New York, NY vacancy
  • Review AI conversations, identify missing context, and suggest follow-up questions that improve...  ...and relevant professional experience. Remote Contract, part‑time Engagement Remote Location...  ...whether they are useful. Conversation evaluators help improve how AI understands context... 
    Remote work
    Contract work
    Part time
    For contractors

    OpenTrain AI, Inc.

    New York, NY
    1 day ago
  • $8 - $65 per hour

    Epidemiology Specialist - Freelance AI Trainer Project United States of America...  ...On a typical day, you will converse with the model on real-world...  ...our prompt engineering and evaluation metrics. A PhD degree in epidemiology...  ...: Contract Workplace type: Remote Seniority level: Mid‑Senior... 
    Remote job
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start

    Meridial

    New York, NY
    3 days ago
  •  ...French Audio Evaluations Specialist - Freelance AI Trainer Project World Wide - Remote Project Overview We are sourcing independent Audio Evaluation Specialists...  ...tasks through simulated interactions, audit conversational AI outputs, and generate clean, reliable... 
    Remote work
    Hourly pay
    For contractors
    Freelance

    Meridial

    United States
    4 hours ago
  •  ...role Meridial seeks independent Audio Evaluation Specialists for an AI benchmark evaluation project. You...  ..., technical troubleshooting, or conversational AI evaluation Native or bilingual proficiency...  ...do not apply Work environment is remote; no office required #J-18808-Ljbffr... 
    Remote work
    Hourly pay
    For contractors
    Freelance
    Work at office
    Worldwide

    remote work

    New York, NY
    5 hours ago
  •  ...plays in travel, finance, and tech support. Evaluate AI audio responses using structured criteria...  ...65 based on experience and location. Fully remote freelance work with flexible scheduling. Meaningful impact on conversational AI quality and reliability. Jobgether Jobgether... 
    Remote job
    Hourly pay
    Freelance
    Flexible hours

    Jobgether

    New York, NY
    1 day ago
  • AuraOne, Inc. is seeking a Domain Contributor Evaluation Specialist for a remote contractor role in the United States. You will review AI outputs against workflows, policy, and stakeholder-fit using a structured rubric, and document the next steps for model training. Responsibilities... 
    Remote job
    For contractors
    10 hours per week

    AuraOne, Inc.

    New York, NY
    1 day ago
  • $45 - $70 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Expert, AI Safety and Model Evaluation Type: Contract Compensation...  ...$45–$70/hour Location: Remote Commitment: 20–40 hours/...  ...Responsibilities Evaluate conversations between users and AI models... 
    Remote work
    Contract work
    Summer work
    Weekday work

    Mercor

    New York, NY
    8 days ago
  •  ...Support [BLS Certification] - Data Conversion - Various Issuers preferred....  ...Constantly (67-100%) Remote Type Onsite Work Location...  ...Avenue Scheduled Weekly Hours 32 Employee Type Regular...  ...These competencies are used to evaluate performance, make hiring decisions... 
    Remote work
    Shift work
    Afternoon shift

    Salem Hospital NSMC

    Swampscott, MA
    12 hours ago
  • $70 - $120 per hour

    $70-120/hr $70-120/hr Mercor Full time, 40 hours...  ...interior visualization specialist who builds and critiques...  ...design data for generative AI training. What you...  ...quality standards “When evaluating an AI‑generated twilight...  ...40 hours a week Where Remote, United States Open to... 
    Remote work
    Full time
    Work at office
    Local area
    Afternoon shift
    Weekday work

    Training Turk

    New York, NY
    3 days ago
  • $40 - $60 per hour

    Visual Content Evaluator (Remote | $40-$60/hr) Position: Visual Content Evaluator Type: Contractor Compensation: $40-$60 per hour Location: Remote...  ...focused on improving the quality and accuracy of AI-generated images. In this role, you will compare original... 
    Remote job
    Hourly pay
    For contractors

    Synthires

    Sacramento, CA
    23 hours ago
  •  ...Multi-Turn Conversation Model Evaluation Specialist is a remote evaluation track for reviewing multi turn conversation model evaluation evaluation prompts and...  ...modeling team can use to retrain. Why this role matters AI data reviewers help turn multi turn conversation... 
    Remote job
    Hourly pay
    For contractors
    10 hours per week

    AuraOne Human Data

    Remote
    9 days ago
  • Prolific is seeking HR and Talent Professionals in Austin, Texas, to join their Expert Network. Your role will involve evaluating AI outputs, developing HR scenarios, and providing feedback on HR policies. Ideal candidates will have professional experience in HR, knowledge... 
    Remote job
    Work from home
    Flexible hours

    Prolific

    Austin, TX
    12 hours ago
  • AuraOne, Inc. is seeking a remote AI Evaluation Rubric Writer reviewer to assess model outputs against a versioned rubric and assign severity tags. You will compare paired responses, label issues, and document recurring failures to help retrain the model. Ideal candidates... 
    Remote job
    For contractors

    AuraOne Human Data

    New York, NY
    1 day ago
  • $25 per hour

    Prolific is seeking AI Training Experts to help train and evaluate cutting‑edge AI models. You’ll complete paid tasks and may be invited to join as a participant after a quick skills test. Tasks include analyzing, editing, and writing annotations, with pay up to $25 per... 
    Remote job
    Hourly pay
    Work from home
    Flexible hours

    Prolific Academic Ltd

    New York, NY
    23 hours ago
  • $20 per hour

    Productive Playhouse Inc. seeks a remote AI evaluation tester to assess chatbots and language models through structured conversations, with occasional voice-based tasks. You will submit deliverables such as write-ups, ratings, screenshots, or recordings as required. Requirements... 
    Remote job
    Flexible hours

    Productive Playhouse Inc.

    New York, NY
    2 days ago
  • $20 per hour

     ...Job Application for Search Engine Evaluation Specialist - Freelance AI Trainer Project at Meridial Back...  ...Independent Contractor Workplace Type: Remote Apply for this job *...  ...ability to participate in professional conversations, explain nuanced concepts, and... 
    Remote work
    Hourly pay
    Part time
    For contractors
    Freelance

    Invisible Technologies

    San Francisco, CA
    a month ago
  • $500 per month

     ...the role The CRO Specialist owns the testing and...  .... They focus on conversion rate, cost-per-lead...  ...results. Location: Remote 9am to 5pm US...  ...Penpot, Framer), using AI design tools to...  ...conversion rate and evaluate lead quality and downstream...  ...sent from our HR/Payroll system,... 
    Remote work
    Local area
    Work from home
    Flexible hours

    Tort Experts

    New York, NY
    5 hours ago
  • German Voice AI Evaluator (Healthcare Front Desk Conversations) About The Role What if your voice could help teach AI how to handle a real phone call to a doctor...  ..., and grade the AI's performance. This is a fully remote, flexible contract role for people with clear speech,... 
    Remote work
    Contract work
    Work at office
    Flexible hours

    Alignerr

    New York, NY
    2 days ago
  • $110k - $138k

     ...and scale intelligent AI, digital, cybersecurity...  ...The Computer Vision Specialist is a member of WWT’s Global...  ...on how models will be evaluated, optimized for edge...  ...carry computer vision conversations, and contribute...  ...Equal Opportunity Employer. LI-DP2 LI-Remote... 
    Remote work
    Full time
    Work experience placement
    Flexible hours
    Shift work

    World Wide Technology

    Edmond, OK
    3 days ago
  • $114.5k - $154.58k

     ...seeking an experienced Principal Conversion Specialist to join our Conversion...  ...Ability to research and evaluate industry and engineering best...  ...recommendations generated through AI-assisted tools, and...  ...engineering productivity #LI-Remote #LI-JC1 Annual salary range... 
    Remote work
    Full time

    Rocket Companies

    Remote
    13 days ago
  • $70 - $120 per hour

    Gridnaut Recruiting is hiring a remote Architectural & Interior Visualization Specialist (AI Evaluation) contractor (pay $70-$120/hr). Contribute to frontier AI research and evaluation work. Ideal candidates: Guide research teams to close knowledge gaps and improve AI... 
    Remote work
    Contract work
    For contractors

    Gridnaut Recruiting

    Remote
    6 days ago
  •  ...(67-100%) Hearing Constantly (67-100%) Remote Type Onsite Work Location 115 Mill Street Scheduled Weekly Hours 32 Employee Type Regular Work Shift...  ...career success. These competencies are used to evaluate performance, make hiring decisions,... 
    Remote work
    Shift work

    Salem Hospital NSMC

    Belmont, MA
    1 day ago
  •  ...work. Today, ServiceNow is the AI control tower for business...  ...vision and requirements for conversational AI features. Turn customer...  ...escalations. ~ Experience designing evaluation datasets, conducting...  ...trust. Work personas (flexible, remote, or required in office) are... 
    Remote work
    Full time
    Work at office
    Immediate start
    Flexible hours

    ServiceNow

    West Palm Beach, FL
    7 days ago
  • $31.2k - $62.4k

     ...to shape the future of AI? Large‑scale language...  ...looking for health sciences specialists who live and breathe...  ...typical day, you will converse with the model on...  ...prompt engineering and evaluation metrics. A bachelor’...  ...Contract Workplace type: Remote Seniority level: Mid... 
    Remote work
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start

    Meridial

    Remote
    3 days ago
  • $8 - $65 per hour

     ...Video Production Specialist - Freelance AI Trainer Project United States of...  ...On a typical day, you will converse with the model on real-world...  ...our prompt engineering and evaluation metrics. A bachelor's or...  ...: Contract Workplace type: Remote Seniority level: Mid-Senior... 
    Remote work
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start

    Meridial

    United States
    23 hours ago
  • $70 - $120 per hour

     ...Architectural & Interior Visualization Specialist (AI Evaluation) is a remote review track for evaluating AI outputs across architectural interior visualization...  ...authorized to work from US. Compensation $70–$120 / hr Application process Apply through AuraOne's... 
    Remote job
    For contractors
    Work experience placement
    10 hours per week

    AuraOne Human Data

    Remote
    2 days ago
  • $8 - $65 per hour

     ...Environment Specialist (Fluent in French) - Freelance AI Trainer Project Switzerland Are...  ...a typical day, you will converse with the model in both...  ...prompt engineering and evaluation metrics. A master's degree...  ...Workplace type: Remote Seniority level: Mid-Senior... 
    Remote work
    Hourly pay
    Contract work
    For contractors
    Freelance
    Immediate start
    Worldwide

    Meridial

    United States
    23 hours ago
  •  ...Literature Specialist – Freelance AI Trainer Project World Wide - Remote Are you a literature expert eager to shape the...  ...models are evolving from simple conversational tools into systems capable of...  ...accuracy and coherence; evaluate the depth and validity of literary... 
    Remote work
    Hourly pay
    Contract work
    For contractors
    Freelance

    Meridial

    United States
    23 hours ago
  • Ecommerce - Conversion Rate Optimization Specialist Remote SUMMARY At Magnolia, the Conversion Rate Optimization Specialist is responsible for improving...  ...observe customer behavior, surface opportunity areas, and evaluate the impact of site improvements. This position works... 
    Remote work
    Work experience placement
    Work at office
    Local area

    Magnolia

    New York, NY
    1 day ago
  • $8 - $65 per hour

     ...Korean–Japanese Bilingual Specialist – AI Trainer World Wide - Remote Are you a bilingual professional fluent in Korean and Japanese...  .... You'll work with cutting-edge AI tools, evaluate translations and conversations between Korean and Japanese, and provide structured... 
    Remote work
    Hourly pay
    For contractors
    Freelance
    Worldwide

    Meridial

    United States
    23 hours ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI Conversation Evaluation Specialist | $32/hr Remote. Be the first to apply!