Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

AI QA Engineer - Test Specialist

$60 - $90 per hour

Mercor

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: QA/Test Engineer
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Design test cases to confirm each task works as intended, including tricky edge cases.
  • Review tasks and reference solutions to catch ambiguity and gaps early.
  • Debug using Python when tasks or checks don't behave as expected.
  • Shape quality checklists and provide actionable feedback to authors.
  • Protect benchmark scores by identifying shortcuts and grading gaps in AI agent runs.

Qualifications

Must-Have

  • MSc or PhD in a STEM field or equivalent practical experience.
  • 1+ years in test engineering, quality assurance, or a research/software engineering role.
  • Skill in designing test cases, quality-review processes, and debugging complex systems.
  • Proficiency in Python and Git .
  • Exceptional attention to detail and clear documentation habits.

Preferred

  • Experience in AI training , model evaluation, or quality review of AI-generated work.
  • A perfectionist mindset with creativity in identifying overlooked issues.

Application Process (Takes 20–30 mins to complete)

  • Upload resume
  • AI interview based on your resume
  • Submit form

Resources & Support

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Vacancy posted 14 days ago
Similar jobs that could be interesting for youBased on the AI QA Engineer - Test Specialist in New York, NY vacancy
  • $30 - $60 per hour

     ...connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our...  ..., Larry Summers , and Jack Dorsey . Position: QA / Software Engineering Reviewer – Browser Test Validation Type: Contract Compensation: $30... 
    Suggested
    Contract work
    Summer work
    Remote work

    Mercor

    New York, NY
    14 days ago
  •  ...Key Responsibilities Train and improve AI models through precise audio data annotation and labeling. Review and validate AI...  ...for quality and accuracy. Provide domain expertise in sound engineering to enhance the performance of AI models. Collaborate on diverse... 
    Suggested
    Hourly pay
    Temporary work
    Flexible hours

    Rise Data Labs

    New York, NY
    a month ago
  •  ...Key Responsibilities Train and improve AI models through detailed audio data annotation and labeling. Review and validate...  ...Required Skills and Qualifications Bachelor’s degree in Sound Engineering, Audio Technology, or a related field. Proven experience in... 
    Suggested
    Temporary work

    Rise Data Labs

    New York, NY
    a month ago
  • $176k

     ...Current PhD, AI Engineering Internship Program - Summer 2027 Key Role Details This is a full-time paid internship program spanning twelve...  ...value and how to orchestrate AI workflows incorporating time-tested systems design principles, integrating both into a unified whole... 
    Suggested
    Full time
    Part time
    Summer work
    Internship
    Local area

    Capital One

    New York, NY
    1 day ago
  •  ...Job Description Job Description Job Title AI QA Engineer Location Hybrid / Remote Employment Type Full-time Job Summary...  ...learning solutions. The ideal candidate will design and execute testing strategies for AI systems, including large language model (... 
    Suggested
    Full time
    Remote work

    Ova Technologies

    New York, NY
    24 days ago
  • We are hiring a Python AI Engineer to operate and evolve the infrastructure behind an enterprise knowledge base platform as it transitions from a Confluence-focused RAG chatbot into a broader, agentic knowledge system. Today, our platform supports Confluence/GitHub ingestion... 

    EPAM Systems

    New York, NY
    3 days ago
  • $300k

     ...comprised of a dynamic group of traders, quantitative researchers, and engineers who work together to examine the global markets, seeking to...  ...research efforts require a fluid mix of skills including AI/ML expertise, engineering pragmatism, statistics, and market intuition... 
    Full time
    Internship
    Visa sponsorship
    Work visa
    Flexible hours

    Jump Trading

    New York, NY
    1 day ago
  • $300k

     ...comprised of a dynamic group of traders, quantitative researchers, and engineers who work together to examine the global markets, seeking to...  ...low-latency inference systems, or pushing the boundaries of AI research from concept to production, you'll have the opportunity... 
    Full time
    Internship
    Visa sponsorship
    Work visa
    Flexible hours

    Jump Trading

    New York, NY
    1 day ago
  • $30 per hour

     ...AI Software Engineer — Python, Document Intelligence & AWS NextGen Coding Company Location: New York City — Hybrid / In-Person Required...  ...authentication, webhooks, Stripe, and third-party services Write tests for OCR, extraction, retrieval, data processing, APIs, and... 
    Contract work
    Immediate start
    Remote work
    Visa sponsorship

    NextGen Coding Company

    New York, NY
    4 days ago
  • We are looking for a Lead Python AI Engineer to build robust, AI-powered API services and agentic workflows. In this role, you will bridge...  ...security, privacy, enterprise-grade observability, and comprehensive test coverage across all backend workflows Contribute actively to... 

    EPAM Systems

    New York, NY
    4 days ago
  •  ...Role Overview We are looking for a highly experienced Senior Backend AI Engineer – Agentic AI to design, develop, and scale distributed backend systems and production-grade Agentic AI solutions. The ideal candidate will have strong expertise in Node.js, TypeScript... 
    Full time

    Neshent Technologies

    New York, NY
    11 days ago
  • $229.9k - $262.4k

    AI Engineer 5 At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One has been...  ...our customers interact with Capital One. Design, develop, test, deploy, and support AI software components including... 
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    3 hours ago
  • $197.3k - $225.1k

    AI Engineer 4 (AI Foundations) Overview: At Capital One, we are creating responsible and reliable AI systems, changing banking for good....  ...how our customers interact with Capital One. Design, develop, test, deploy, and support AI software components including... 
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    3 hours ago
  • $161.8k - $184.6k

    AI Engineer 3 (AI Foundations) At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years,...  ...how our customers interact with Capital One. Design, develop, test, deploy, and support AI software components including... 
    Full time
    Part time
    Work at office
    Local area

    Capital One

    New York, NY
    3 hours ago
  • Location: New York; USAWork type: RemoteSeniority: SeniorPrimary skill: Python.CorePosted: 2026-11-05We are hiring a Python AI Engineer to operate and evolve the infrastructure behind an enterprise knowledge base platform as it transitions from a Confluence-focused RAG... 
    Temporary work

    EPAM Systems

    New York, NY
    4 days ago
  • $314.8k - $359.3k

    Sr. Staff AI Engineer At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital...  ...how our customers interact with Capital One. Design, develop, test, deploy, and support AI software components including... 
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    3 hours ago
  •  ...AI / Forward Deployed Engineer Requires 4+ years of full-time professional experience after completing a degree. Candidates must have existing US work authorization. H1B transfers and TN status are accepted. No new visa sponsorship, OPT, or CPT. We are an executive... 
    Full time
    H1b
    Visa sponsorship

    Long Ridge Partners

    New York, NY
    2 days ago
  • $130k - $180k

     ...Payments verticals, supplemented with Data & AI, Cybersecurity, Risk & Compliance, Change...  ...seeking a hands-on Forward Deployed AI Engineer to work directly with business...  ...and solution design through development, testing, deployment, and ongoing optimization.... 
    Temporary work
    Remote work
    Worldwide

    BIP US

    New York, NY
    3 days ago
  • $269.1k - $307.2k

    Staff AI Engineer At Capital One, we are creating responsible and reliable AI systems, changing banking for good. For years, Capital One...  ...how our customers interact with Capital One. Design, develop, test, deploy, and support AI software components including... 
    Full time
    Part time
    Local area

    Capital One

    New York, NY
    3 hours ago
  • $244.7k - $279.2k

     ...Director AI Engineering (Remote Eligible) At Capital One, we are creating responsible and reliable AI systems, changing banking for good...  ...interact with Capital One. Oversee the design, development, testing, deployment, and operation of the platform's core systems: distributed... 
    Full time
    Part time
    Local area
    Remote work

    Capital One

    New York, NY
    3 hours ago
  •  ...Our client, one of the world's leading law firms, is hiring an AI Legal Engineer to help lead its firmwide AI transformation. The firm advises...  ...and automation Quality, risk and governance Define testing and evaluation frameworks to measure accuracy, reliability and... 
    Contract work

    Coda Search¦Staffing

    New York, NY
    22 days ago
  • $170k - $300k

     ...AI Engineer | $10B+ Consumer FinTech | NYC $170k–$300k Base + Bonus + Equity New York City | 5 Days Onsite I'm partnering with a $10B+ consumer fintech platform that's rethinking how people earn rewards and interact with the places they live, shop and spend.... 
    Local area

    Orbis Group

    New York, NY
    16 days ago
  • $180k - $200k

     ...Forward Deployed AI Engineer New York, NY (On-site, Full-Time) We're partnered with a premier global investment and technology firm that is making a significant investment in AI as a core business capability. As part of that effort, they're hiring Forward Deployed... 
    Full time
    For contractors

    Selby Jennings

    New York, NY
    16 days ago
  •  ...Job Title:- Anthropic Claude AI Engineer / Claude Developer Location:- New York City (4 days/week onsite) Overall experience – 10+ years...  ...and tools Develop and prototype enterprise GenAI use cases Test, troubleshoot and productionize Claude-based solutions... 

    Envision Technology Solutions

    New York, NY
    1 day ago
  • $350k

     ...Senior AI Engineer sought to join a small, exceptionally fast-moving engineering team building an AI-native consumer platform. There is a salary up to $500K + equity on offer here, so we are only interesting in exceptional candidates with a genuine production AI development... 
    Full time

    IT Recruitment Solutions

    New York, NY
    4 days ago
  • $98.3k - $135.2k

     ...Senior and Lead Software Engineers (AI Engineering, .NET, Azure) Chicago, Dallas, Houston or New York (Hybrid) Full-time, direct hire...  ...confidential data Harden internally built prototypes into secure, tested, maintainable software Work directly with non-technical... 
    Full time
    Internship
    Relocation package

    1872 Consulting

    New York, NY
    2 days ago
  •  ...AI Engineer (GenAI Workflows for OTC Derivatives) Location: New York City (SoHo) — In-Person About Our Client Our client is building...  ...of both traditional SDLC and AI SDLC (versioning, testing, rollout, monitoring) Strong software design fundamentals... 
    Full time

    Nxt Level

    New York, NY
    1 day ago
  •  ...AI Engineer – Agentic AI / Generative AI Location: Mountain View, CA or New York, NY (Hybrid – 3 days onsite weekly) Employment Type: Contract (6 months with potential extensions) About the Opportunity Optomi is seeking an experienced AI Engineer for an exciting... 
    Contract work
    3 days per week

    Optomi

    New York, NY
    3 hours ago
  • $60 - $70 per hour

     ...Immediate need for a talented AI Engineer . This is a 06+ Months Contract opportunity with long-term potential and is located in Mountain View, CA , or New York City, NY (Hybrid) . Please review the job description below and contact me ASAP if you are interested.... 
    Contract work
    Local area
    Immediate start

    Pyramid Consulting, Inc

    New York, NY
    15 days ago
  • $110 - $150 per hour

     ...place. Our client, an investment firm, is seeking a Senior AI Engineer to join their team in New York, NY! This role is ideal for...  ...and best practices for AI development, code quality, testing, and deployment. Mentor engineers through technical guidance... 

    Eleven Recruiting

    New York, NY
    a month ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to AI QA Engineer - Test Specialist. Be the first to apply!