Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

QA Engineer - AI Test Specialist

$60 - $90 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: QA/Test Engineer
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Design test cases to confirm task functionality, including edge cases.
  • Review tasks and reference solutions to catch ambiguities and gaps early.
  • Debug using Python when tasks or checks misbehave.
  • Build quality checklists and provide actionable feedback to authors.
  • Monitor AI agent runs for shortcuts and grading gaps to ensure trustworthy benchmark scores.

Qualifications

Must-Have

  • MSc or PhD in a STEM field or equivalent practical experience.
  • 1+ years in test engineering, quality assurance, or a related role.
  • Experience designing test cases and debugging complex systems.
  • Proficiency in Python and Git .
  • Exceptional attention to detail and clear documentation habits.

Preferred

  • Experience in AI training , model evaluation, or quality review of AI-generated work.
  • A perfectionist mindset and ability to work independently through ambiguous problems.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the QA Engineer - AI Test Specialist in San Francisco, CA vacancy
  •  ...QA Engineer Location: Candidate should be local to below locations and should be ready to work from office: Alpharetta, GA Rancho, CA Oakland...  ...engineers in an Agile team. Design, develop and review test strategies and test cases for the applications. Analyze new developments... 
    Suggested
    Work at office
    Local area
    Flexible hours

    Yochana

    San Francisco, CA
    1 day ago
  •  ...on a mission to create the world’s first AI-powered Personal & Entrepreneurial Resource...  ...along the way. The Role As a QA Engineer at Air Apps, you will be responsible for...  ...applications through manual and functional testing . You will work closely with developers,... 
    Suggested
    Temporary work
    Worldwide

    Air Apps

    San Francisco, CA
    1 day ago
  • $140k - $180k

     ...AI Imaging Specialist - Radiology Latch builds rigorous scientific benchmarks for AI agents across...  ...each case and build the studies that test it, from whether a scan was acquired...  ...interpretation and run scripts. No specific ML engineering is required but could come in handy... 
    Suggested
    Full time
    For contractors
    H1b
    Work at office
    Remote work
    Night shift

    LatchBio

    San Francisco, CA
    12 hours ago
  • $170k - $200k

     ...advertising through the intelligent application of AI and machine learning, Tatari helps some...  ...on TV. We are looking for a Staff QA Engineer to join the Buying and Optimization team....  ...leadership to raise the bar for testing practices, set best practices, and scale... 
    Suggested
    Contract work
    Work at office
    2 days per week

    Tatari

    San Francisco, CA
    4 days ago
  •  ...Purple Drive QA / Test Engineer Share Contractual San Francisco, CA PDT - 8148 8-10 Overview: QA / Test Engineer Requirement Gathering Develop and implement SAP testing strategies Manage testing teams Coordinate with stakeholders We are seeking a highly... 
    Suggested

    Purple Drive

    San Francisco, CA
    2 days ago
  • $80 - $120 per hour

     ...Job Description Job Description About the job Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers... 
    Contract work
    Summer work
    Work at office
    Remote work

    Mercor

    San Francisco, CA
    29 days ago
  • $62 per hour

     ...creative and technical talent with leading AI research labs. Headquartered in San...  ...benchmarks, and playbooks to maintain consistent testing. Document reproducibly by producing...  ...testing, exploit development, reverse engineering. Skills in Socio-technical risk : harassment... 
    Remote job
    Contract work
    Summer work

    Mercor

    San Francisco, CA
    25 days ago
  •  ...Job Description Job Description We are looking for an Artificial Intelligence (AI) Specialist: Forge the Next Chapter in AI Advancements   Key Qualifications:   Original Contributions: Your innovations in AI, especially within the ChatGPT technology sphere... 

    Forhyre

    San Francisco, CA
    4 days ago
  • Remote is seeking a Business and Management Specialist for a freelance AI trainer project. You will engage the model with business scenarios and strategic exercises, assessing realism and usefulness while documenting errors and suggesting improvements. Ideal candidates... 
    Remote job
    Hourly pay
    Contract work
    Freelance
    Flexible hours

    Remote

    San Francisco, CA
    2 days ago
  • $66 - $84 per hour

     ...technical talent with leading AI research labs. Headquartered...  ...Applied Computer Science Benchmark Specialist Type: Contract...  ...computer science questions that test deep conceptual understanding...  ...Science , Electrical Engineering , or a closely related field... 
    Contract work
    Summer work
    Remote work

    Mercor

    San Francisco, CA
    13 days ago
  • Remote is seeking a Photoshop Specialist - AI Trainer to help shape the future of AI through expert image editing. You will work remotely...  ...layers, and color workflows. The role emphasizes documentation of failures and prompt engineering insights. #J-18808-Ljbffr Remote
    Remote job
    For contractors

    Remote

    San Francisco, CA
    12 hours ago
  • TSMG is seeking Quality control specialists to support an AI language project aimed at improving speech recognition systems. As a QC contributor, you will review completed language annotation tasks and ensure AI transcripts are natural, readable, and aligned with project... 
    Remote work

    Remote Services Inc.

    San Francisco, CA
    1 day ago
  • About Kastle Kastle is building the AI operating system for consumer lending . We partner with some of America’s largest mortgage lenders...  ...by providing feedback on conversations, annotating data, and testing for quality. If you have worked as a customer service or... 
    Part time
    Remote work
    Work from home
    10 hours per week
    Flexible hours

    Kastle

    San Francisco, CA
    12 hours ago
  • Logoslabs seeks a Swahili Language Specialist to help train and evaluate AI models from home as a contract remote expert. You will test Swahili language capabilities, identify errors, and document outcomes to guide model improvements. The role emphasizes grammar, morphology... 
    Remote job
    Contract work
    For contractors
    Flexible hours

    Remote

    San Francisco, CA
    12 hours ago
  • $35 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...Follow taxonomies, benchmarks, and playbooks to ensure consistent testing. Document reproducibly. Produce reports, datasets, and attack... 
    Remote job
    Contract work
    Summer work

    Mercor

    San Francisco, CA
    29 days ago
  • $145k - $187k

     ...molecular diagnostics, digital health solutions, AI-powered data and analytics, and patient...  ...lives. The Senior Data and AI Engineer will serve as a senior hands-on technical...  ...normalize diseases, diagnoses, laboratory tests and results, demographics, signs and symptoms... 
    Full time
    Local area
    Remote work
    Worldwide

    CareDx, Inc.

    Brisbane, CA
    2 days ago
  • $138k - $200k

     ...Intelligence Sales SpecialistAs an Artificial Intelligence (AI) Sales Specialist, you will help us grow our AI business by building and...  ...Google accounts and cross-functional teams (e.g. Customer Engineering, Marketing, Customer Success, Product, Engineering, Channels... 

    CompliancePoint

    San Francisco, CA
    12 hours ago
  • $140k - $153k

     ...optimized hardware, software, and increasingly AI, coupled with the patient-centric...  ...late again". We are looking for a Senior QA Engineer I to join our team! You will be...  ...product managers and translate them into test strategies. Create, execute, and maintain... 
    Local area
    Immediate start
    Remote work
    Flexible hours

    Prenuvo

    San Francisco, CA
    17 days ago
  • $170k - $205k

     ...intelligence . As the only vertically integrated AI infrastructure company built from the...  ...You'd be the first dedicated quality engineer on the team, and the person who defines what...  ...practice. What You'll Be Working On: Test Strategy Ownership: Own end-to-end test... 
    Contract work
    Temporary work

    Crusoe

    San Francisco, CA
    12 days ago
  • $45 - $50 per hour

     ...(Expert), SQL/MySQL (Advanced), JIRA (Intermediate), Agile Testing (Intermediate) Contract Type: W2 and C2C Location: Fremont...  ...Job Summary: We are seeking an experienced Software QA Automation Engineer to join the Quality Assurance team, responsible for... 
    Hourly pay
    Contract work
    Work at office

    Akraya, Inc.

    San Francisco, CA
    21 days ago
  •  ...leaders in helping drive that change, with strong ecosystem relationships. We combine our strength in technology and leadership in cloud, AI and data with unmatched industry experience, functional expertise, and global delivery capability. We are uniquely able to deliver... 
    Work experience placement
    Live in
    Work at office
    Local area

    Accenture

    San Francisco, CA
    1 day ago
  • $138k - $200k

     ...Artificial Intelligence Sales Specialist As an Artificial Intelligence (AI) Sales Specialist, you will help us grow our AI business by building and...  ...accounts and cross-functional teams (e.g. Customer Engineering, Marketing, Customer Success, Product, Engineering,... 

    CompliancePoint

    San Francisco, CA
    1 hour ago
  •  ...Job Application for Literature Specialist – Freelance AI Trainer Project at Meridial Back to jobs Literature Specialist – Freelance AI Trainer Project World Wide - Remote Are you a literature expert eager to shape the future of AI? Large-scale language models... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Freelance
    Remote work

    Invisible Technologies

    San Francisco, CA
    a month ago
  • $50 - $60 per hour

     ...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors...  ...interviewing, or technical-vetting experience. Comfort assessing senior engineers. Strong communication and structured judgment. Weekday... 
    Contract work
    Summer work
    Remote work
    Weekday work

    Mercor

    San Francisco, CA
    a month ago
  •  ...Alignment & Resource Partner (Nepali) - Freelance AI Trainer Project at Meridial Back to...  ...native-level Nepali language vetting and QA for a specialized AI data project. As AI...  ...& Review: Reviewing, annotating, and testing AI outputs for grammatical accuracy, naturalness... 
    Hourly pay
    Part time
    For contractors
    Freelance
    Remote work

    Invisible Technologies

    San Francisco, CA
    a month ago
  • $10 - $30 per hour

     ...Job Application for Information Specialist - Freelance AI Trainer Project at Meridial Back to jobs Information Specialist - Freelance AI Trainer Project United States of America Project Overview We are sourcing independent Information Specialists to provide... 
    Hourly pay
    Part time
    For contractors
    Freelance
    Remote work

    Invisible Technologies

    San Francisco, CA
    a month ago
  • $8 - $65 per hour

     ...Job Application for Korean-Japanese Bilingual Specialist - Freelance AI Trainer Project at Meridial Back to jobs Korean-Japanese Bilingual...  ...linguistic guidelines. Tasks may include verifying terminology, testing conversational flow, and ensuring accurate rendering of... 
    Hourly pay
    Contract work
    Part time
    For contractors
    Freelance
    Remote work
    Worldwide

    Invisible Technologies

    San Francisco, CA
    a month ago
  •  ...interested in business and management and eager to shape the future of AI? Large-scale language models are evolving from simple...  ...insurance and PTO do not apply. Job title: Business and Management Specialist - Freelance AI Trainer Project Employment type: Freelance / Contract... 
    Hourly pay
    Contract work
    For contractors
    Freelance
    Remote work

    Remote

    San Francisco, CA
    2 days ago
  • $8 - $65 per hour

    Are you a sound engineering expert eager to shape the future of AI? Large-scale language models are evolving from clever chatbots into powerful tools for...  ...generation of AI.We're looking for sound engineering specialists who live and breathe live sound reinforcement,... 
    Hourly pay
    For contractors
    Remote work

    Remote Services Inc.

    San Francisco, CA
    3 days ago
  • Synthires is seeking a detail-oriented Generalist - Research & AI Evaluation for remote, part-time contract. You will develop research questions and evaluation materials to benchmark advanced AI models, and you will gather and summarize information across diverse subjects... 
    Remote job
    Contract work
    Part time
    Immediate start

    Synthires

    San Francisco, CA
    2 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to QA Engineer - AI Test Specialist. Be the first to apply!