QA Engineer - AI Test Specialist
$60 - $90 per hourMercor
Job Description
Job Description
About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .
Position: QA/Test Engineer
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week
Role Responsibilities
- Design test cases to confirm task functionality, including edge cases.
- Review tasks and reference solutions to catch ambiguities and gaps early.
- Debug using Python when tasks or checks misbehave.
- Build quality checklists and provide actionable feedback to authors.
- Monitor AI agent runs for shortcuts and grading gaps to ensure trustworthy benchmark scores.
Qualifications
Must-Have
- MSc or PhD in a STEM field or equivalent practical experience.
- 1+ years in test engineering, quality assurance, or a related role.
- Experience designing test cases and debugging complex systems.
- Proficiency in Python and Git .
- Exceptional attention to detail and clear documentation habits.
Preferred
- Experience in AI training , model evaluation, or quality review of AI-generated work.
- A perfectionist mindset and ability to work independently through ambiguous problems.
Application Process (Takes 20–30 mins to complete)
- Upload resume
- AI interview based on your resume
- Submit form
Resources & Support
- For details about the interview process and platform information, please check:
- For any help or support, reach out to: View email address on ziprecruiter.com
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Vacancy posted 10 days ago
Similar jobs that could be interesting for youBased on the QA Engineer - AI Test Specialist in San Francisco, CA vacancy
$65 per hour
Mercor, located in San Francisco, is searching for a Private Banking Expert to evaluate and enhance AI systems within the private banking sector. This role involves collaborating with research teams to improve AI models and provide critical feedback to refine data quality...SuggestedRemote jobHourly payFlexible hours- ...is seeking a skilled professional to work on their enterprise AI platform, which automates voice sales conversations for healthcare... ...experience in LLM-powered applications, particularly in prompt engineering, proficient in Python and TypeScript, and a solid understanding...Suggested
$90k - $150k
...as we help shape a brighter way forward. QA/QC Engineer (MEP/Cx) Data center project field QA/QC... ...track all required QA/QC deliverables and test reports during L2 and L3 (NETA, Short Circuit... ...the power of artificial intelligence (AI) to efficiently accelerate meaningful...SuggestedFull timeWork experience placementFor subcontractorLocal area$80k - $100k
...'s first accessible X-Ray CT scanner for engineers. Our easy-to-use scanner and cloud-based... ...unprecedented visibility into products, as well as AI-driven tools that highlight problems and... ...the role: Lumafield is looking for a QA Test Engineer to help ensure the quality and...SuggestedFull timeWork at officeWork visaFlexible hoursLumafield
San Francisco, CA1 day ago$152.9k - $210.2k
...range of consumer experiences and devices.As a Staff QA Engineer in the Dolby Cinema QA group, you will provide senior test leadership for next-generation cinema imaging... ...external hardware partners and help advance practical AI-assisted QA tools that improve test efficiency...SuggestedFull timeWork at officeLocal areaWorldwide$204.44k - $324.99k
...requirements, and participate in code reviews, testing, and deployment activities using CI/CD... ...Prompt Builder templates and grounded AI experiences using Salesforce data, Data Cloud... ...I and Salesforce Certified Agentforce Specialist are required. Platform Developer II,...H1bLocal area- ...revolutionizing software development with AI-powered formal verification. We've... ...About the role Join our team as an AI Engineer and help us push the boundaries of what's... ...with a talented team of AI experts, EBM specialists, formal verification engineers, and software...Full timeContract work
Logical Intelligence
San Francisco, CA1 day ago - ...QA Engineer Location: Candidate should be local to below locations and should be ready to work from office: Alpharetta, GA Rancho... ...engineers in an Agile team. Design, develop and review test strategies and test cases for the applications. Analyze new...Work at officeLocal areaFlexible hours
Yochana
San Francisco, CA5 days ago - ...sales discovery to POCs, architecting tailored solutions and driving measurable outcomes. You will collaborate with Sales, Product, Engineering, and Marketing, conduct live demonstrations, deliver custom proposals, and build reusable assets to accelerate adoption. #J-1880...
$148.5k - $177.4k
Fin is the AI Customer Agent company on a mission to help businesses provide perfect customer experiences.Our AI Agent Fin is the highest... ...their impact with AI.You’ll partner with a dedicated team of engineers for complex technical work while managing the overall project...Work at officeLocal areaWork from home3 days per week$141.6k - $177k
Scale AI's Advisory practice is the most forward-leaning bet in our go-to-market motion... ...that shape the work, and pressure-testing the team's assumptions. You also run the... ...trackers, and coordination across Solutions Engineering and Design, and acting as the single source...Full timeWork experience placement$82.6k - $162.8k
Position Summary We are seeking an AI Governance and Privacy Consultant who can... ...governance checkpoints into product and engineering delivery through architecture reviews,... ...evaluation practices, including prompt injection testing, jailbreak resilience, hallucination...Local areaVisa sponsorship$105.4k - $207.8k
...foundations that power the next generation of AI-enabled cyber defense?If yes, then... ...for you.We are looking for a hands-on Data Engineer to build and operate the governed data foundations... ...and defects, risk acceptance, control testing outcomes, audit evidence, and policy...Local areaVisa sponsorship$100k - $160k
Applied AI Consultant: Build, Teach, Transform.Location: London, New York, San FranciscoTravel... ...(which too many firms offer) and AI engineering (which too many firms keep siloed). Youll... ...own operations first, giving you battle-tested credibility with every client...Full timeFlexible hoursShift work$25 per hour
...creative and technical talent with leading AI research labs. Headquartered in San... ...benchmarks, and playbooks to ensure consistent testing. Document reproducibly to produce... ...penetration testing, exploit development, reverse engineering. Expertise in socio-technical risk:...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$90 per hour
...Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors... ..., Larry Summers , and Jack Dorsey . Position: Software Engineering Expert Type: Contract Compensation: $60–$90/hour...Contract workSummer workRemote workMercor
San Francisco, CA2 days ago$144.5k - $230k
...AI Data Specialist DevExplore San Francisco, CA AI Data Specialist Use AI to assess how you fit Get AI-powered advice on this... ...Intelligence Lab The AI Lab is composed of leading software and AI engineers designing agentic-AI solutions. The group builds and...Full timeWork at officeLocal areaRemote workShift work2 days per weekMMI Textiles, Inc.
San Francisco, CA4 days ago- Mercor is building a red team to probe AI models with adversarial inputs, surface vulnerabilities, and generate red team data that strengthens safety for customers. This is a text-based, higher-sensitivity project where participation is optional and guided by clear guidelines...Remote job
$200k - $240k
...publicly traded technology leader, is looking for its first AI Innovation Specialist to join the corporate Law Department. This newly created,... ...background demonstrating technical fluency, such as a CS, Engineering, or Math degree, or equivalent technical experience....$70 per hour
We are seeking experienced AI Safety Practitioners to evaluate the safety, quality, and alignment of frontier AI models across complex... ...across nuanced and high-impact domains. Collaborate with leading AI researchers, engineers, and safety teams. #J-18808-Ljbffr ObsidianRemote jobWorldwide$146.4k - $235.38k
...contract lifecycle management (CLM). What you’ll do The Senior AI Specialist - Partner Digital Transformation & Success plays a pivotal... ...AI-driven partner transformation Support field, product, and engineering teams to ensure seamless execution of AI initiatives globally...Permanent employmentFull timeContract workWork at officeLocal areaRemote work2 days per week- Mercor is offering a remote independent contractor position focused on evaluating AI models for visual document understanding. The role involves authoring complex tasks with a clear ground-truth output and objective rubric. It allows for a flexible schedule, requiring...Remote jobWeekly payFor contractorsFlexible hours
- AT Recruitment (ATR) is seeking a Forward Deployed Generalist — Early Career in San Francisco. You will own deployments end to end, translate customer needs into live configurations, and drive rapid time-to-value. The role sits at the intersection of product, customer go...
$35 per hour
...Exists At Mercor, we believe the safest AI is the one that’s already been attacked —... ...taxonomies, benchmarks, and playbooks to keep testing consistent Document reproducibly:... ...penetration testing, exploit development, reverse engineering Socio-technical risk: harassment/disinfo...Remote job$70.35k - $196k
...the elite technical and product engine of the Accenture Google... ...technology: the move to Agentic AI and Product‑Led Operating Models... ...Are A passionate, hands‑on GCP specialist/practitioner who understands that... ...on Vertex AI as you are testing a solution built by other team...Work experience placementLive inWork at officeLocal areaShift work- BairesDev is seeking a CRM Senior Specialist to own CRM execution for VIP Audiences & Experiences, building AI-assisted outreach sequences across clients, former clients, and event audiences. You will leverage Claude daily for copy, research, intelligence, and workflow...Remote job
$1,750 - $2,150 per month
Obsidian is looking for experienced cybersecurity professionals to review AI systems' threat detection and vulnerability assessments. Responsibilities include evaluating AI outputs and creating realistic cybersecurity scenarios. Ideal candidates should have over 3 years...- ...linking, and accounting integrations. You’ll work with Engineering, Risk, Product, and Partnerships to resolve complex problems and improve workflows. You’ll use AI tools, build automations, and uphold high CSAT/QA standards while communicating clearly with customers, often...
- Mercor is building a fully remote red team to probe AI models for safety, resilience, and ethical considerations. You will work on adversarial inputs, jailbreaks, and vulnerability discovery while producing actionable data and reproducible reports for customers. This role...Remote job
- ...Welo Data is seeking a Data Labeling Associate for Project Perseus in San Francisco, CA. This position involves critiquing Arabic AI outputs for accuracy, identifying biases in datasets, and ensuring model improvement. Candidates must have native Turkish proficiency,...
Welo Data
San Francisco, CA5 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to QA Engineer - AI Test Specialist. Be the first to apply!
Related searches
- qa automation test engineer San Francisco, CA
- qa engineer San Francisco, CA
- qa automation engineer remote San Francisco, CA
- software test engineer San Francisco, CA
- software quality assurance engineer San Francisco, CA
- sdet qa automation engineer San Francisco, CA
- software quality engineer San Francisco, CA
- quality assurance engineer San Francisco, CA
- senior software test automation engineer San Francisco, CA
- qa specialist San Francisco, CA