AI Quality Engineer
Expedite Technology Solutions LLC
Job Title
Key Responsibilities
- Test Generative AI and conversational systems
- Perform LLM evaluation and benchmark testing
- Validate RAG (retrieval quality, relevance, grounding)
- Build automation frameworks (Playwright/Selenium)
- Conduct API, regression, and E2E testing
- Perform AI safety, red teaming, and bias validation
- Support observability, monitoring, and HITL workflows
- Collaborate with engineering, QA, and DevOps teams
- Support CI/CD quality pipelines
Required Skills
- QA & automation testing
- Playwright/Selenium, API testing (Postman, REST)
- GenAI & RAG validation, LLM evaluation
- Prompt engineering, conversational AI testing
- HITL testing, AI safety & observability
- SQL, Git, Python, CI/CD concepts
Preferred
- OpenAI, Azure OpenAI, ChatGPT, Claude
- LangChain, CrewAI, LangGraph, MCP
- Vector DBs (Pinecone, ChromaDB, Weaviate)
- RAGAS, DeepEval
- Python/JS, Docker, Kubernetes, GitHub Actions
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the AI Quality Engineer in United States vacancy
$157k - $249k
Job TitleSenior Data Quality & Active Learning AI EngineerJob DescriptionSenior Data Quality & Active Learning AI EngineerBuild compliant, audit... ...and development, working closely with other data and AI engineers, medical informaticians, architects, data stewards, clinical...SuggestedFull timeWork at officeImmediate startRelocation package3 days per week- ...media, and sports—by leveraging data and AI as core assets. Our AI-first, cloud-native... ...is seeking a Senior or Staff AI Software Engineer in Test to join our AI Engineering team... ...data scientists, contributing production-quality code to shared repositories. The ideal candidate...SuggestedFull timeContract work
Twg Global Ai
Remote1 day ago - ...ID: 27935034Reference Number: 26-00337Title: Gen Ai QA EngineerPosted Date: 2026-04-08Contact: Asif RazaContact... ..., document reproducible steps, and partner with engineers to resolve issues.Contribute to release readiness, quality gates, and continuous improvement of testing...SuggestedLocal area
$221k - $247k
...Learn more about ourTotal Rewards philosophy.AI is a fundamental part of how work gets... ..., accelerate cycle times, and improve quality and risk outcomes.Align roadmap priorities... ...Applications EnablementPartner with EAIT engineers, administrators, AI architects, and platform...SuggestedFull timeContract workWork at officeLocal areaShift work2 days per week3 days per week$80k - $140k
Quality Assurance Engineer - AI Testing, Charles River Development/State Street AlphaWhy this role is important to usThe team you will be joining is responsible for advancing quality engineering for AI-enabled capabilities within the CRIMS / State Street Alpha product suite...SuggestedFull timeTemporary workFlexible hours$160k - $180k
...inspired each day to stretch, grow, and make an impact.Learn about the Danaher Business System which makes everything possible.The AI Quality Engineer leads quality assurance efforts for AI‑driven medical device software, overseeing end‑to‑end lifecycle development and...Full time$156.5k - $290.1k
....In this role you will lead the strategy, operationalization, governance alignment, and continuous evolution of enterprise AI Quality Engineering capabilities across Vizient. You will establish scalable AI Quality Engineering operating models, validation frameworks, runtime...Full timeFor contractors- ...Deposco is seeking an experienced AI Quality Engineer to join our innovative quality assurance team. This role focuses on ensuring the accuracy, reliability, and performance of AI-driven applications, models, and automation workflows. The ideal candidate will be highly...Full timeWork at office
Deposco
Alpharetta, GA5 days ago - ...Quality AI Engineer Location : Remote ( Candidates must be available for an in-person interview or laptop pickup at one of the following locations: San Francisco, CA | Arlington, VA | Denver, CO | Chicago, IL | Boston, MA | New York, NY | Houston, TX | Miami, FL...Local areaRemote work
Akaasa Technologies
Remote1 day ago $33 - $36 per hour
...Quality Analyst Our client is seeking a Quality Analyst to join their team. Job Title... ..., automated testing workflows, and AI-assisted quality frameworks that increase... ...~ Bachelor's degree in computer science, Engineering, Data Science, Information Systems, AI, or...Weekly payTemporary workRemote workFlexible hoursExperis
United States1 day ago- ...Seeking an experienced AI Quality Engineer for a contract position, who will lead the validation, certification, and production readiness of enterprise Generative AI and AI-powered automation solutions, working remotely while collaborating with cross-functional teams to...Contract workRemote work
Virtual Vocations Inc
United States1 day ago $20 per hour
...A leading AI training company in the United States is seeking an Audit Defense Specialist to evaluate and improve AI chatbots in healthcare... ...issues for AI chatbots and ensuring output accuracy and quality, with flexible and remote working options. Compensation starts at...Hourly payRemote workFlexible hoursDataAnnotation
Helena, MT1 day ago$20 per hour
A technology firm in Michigan is seeking an Audit Defense Specialist to improve AI models by providing them with healthcare-related challenges. This role requires expertise in fields such as Revenue Cycle Management, Pharmacy Operations, and other healthcare backgrounds...Hourly payFor contractorsRemote workDataAnnotation
Lansing, MI5 days ago- ...Key Responsibilities: Design robust test scenarios to validate AI agent behavior and performance. nalyze Product Requirement Documents (PRDs) to extract testing requirements and ensure comprehensive coverage. Conduct performance testing of AI systems...Local area
Macpower Digital Assets Edge
Brooklyn, NY5 days ago $20 per hour
...A data modeling company is seeking an Audit Defense Specialist to help train AI models by measuring progress and addressing AI chatbot issues. This role requires expertise in healthcare and relevant backgrounds such as Revenue Cycle Management or Pharmacy Operations....Hourly payFor contractorsRemote workDataAnnotation
New York, NY5 days ago$88.9k - $155.5k
...inclusive environment both now and in the future.SummaryIn this role, you will lead the day-to-day execution and delivery of AI Quality Engineering initiatives supporting Vizient's enterprise AI transformation efforts. You will help establish and scale AI Quality...Full timeFor contractors$20 per hour
A healthcare-focused technology company is seeking an Audit Defense Specialist to enhance AI models. You will provide complex healthcare problems to AI, assess outputs for accuracy, and ensure medical precision. Candidates should have a degree in a healthcare field and...Hourly payFor contractorsRemote workDataAnnotation
Brooklyn, NY1 day ago$20 per hour
...A leading AI training company is seeking an Audit Defense Specialist to help train AI models by providing complex healthcare problems... ...schedule and hourly pay starting at $20+, plus bonuses for quality work. Only applicants in the United States will be considered. #J...Hourly payFull timePart timeRemote workFlexible hoursDataAnnotation
Wyoming, OH5 days ago$20 per hour
A leading AI training organization is seeking analytical individuals for remote roles focusing on training AI chatbots. Responsibilities include developing varied conversation prompts, writing quality responses, evaluating model performance, and fact-checking. Ideal candidates...Hourly payRemote workFlexible hoursDataAnnotation
Louisiana, MO2 days ago- ...Summary ALP-U is hiring a contract QA engineer to embed in the team as a full sprint member. The role provides a quality pass on completed work before promotion to the... ...based test prioritization. Proficiency with AI tooling (Claude, Claude Code, or equivalent) integrated...Contract work
Cortex consultants LLC
Sunnyvale, CA1 day ago - ...About The Role We are looking for a highly skilled AI QA Engineer to join our team and ensure the quality, reliability, and performance of AI/ML-driven applications. The ideal candidate will have a strong background in software testing, a solid understanding of AI/...
Ova Technologies
Alpharetta, GA2 days ago - ...Passionate about transforming software quality, the full-time AI QA Engineer will design and implement AI-powered testing agents, build intelligent testing workflows, and optimize testing strategies while collaborating with engineering teams in a dynamic environment. Key...Full timeRemote work
Virtual Vocations Inc
United States1 day ago - ...QA Engineer – Agentic ERP Platform The QA Engineer is responsible for hands-on testing and quality assurance of Rimini Street's Agentic ERP Platform, ensuring that AI-driven features meet functional requirements, performance standards, and enterprise reliability expectations...Contract workWork at officeRemote work
Rimini Street
United States5 days ago - A leading technology company is seeking a Senior QA Experimentation Engineer to join their AI Innovation team. This remote role involves defining QA strategies, validating AI-driven products, and collaborating within Agile teams to deliver reliable solutions. Candidates...Remote work
WillDom Global, LLC.
Clay, NY1 day ago - ...Job Title AI QA Engineer Location Hybrid / Remote Employment Type Full-time Job Summary We are seeking an AI QA Engineer to ensure the quality, reliability, safety, and performance of AI-powered applications and machine...Full timeRemote work
Ova Technologies
New York, NY2 days ago - ...Description Job Description ASSYST is seeking a Senior AI Automation Test Engineer in Washington D.C. (Remote role). This role involves... ...engineering with a strong emphasis on AI‑driven quality assurance. At its core, it combines traditional automation...Local areaRemote work
ASSYST, Inc.
Washington DCmore than 2 months ago - ...Job Description We are seeking a Software Automation Quality Assurance Engineer with experience in healthcare software testing to ensure the... ...: Experience with CCM, RPM & RTM healthcare platforms AI-assisted Testing Tools & QA Practices HL7, FHIR & EHR Integrations...Work from homeFlexible hours
Symphony Solutions
Cumming, GA16 days ago $145k - $165k
Transformation Engineer - Software Development, AI & Quality EngineeringFitch Group is seeking a hands-on Transformation Engineer with strong expertise in software development, AI-enabled engineering, and modern quality practices. This role will focus on building intelligent...Work at officeImmediate startWorldwideFlexible hoursShift work$135.2k - $236.6k
...performance through an inclusive environment both now and in the future.Summary:In this role, you will lead Quality Engineering and product validation for an AI-enabled healthcare operations analytics product that integrates EHR, ERP, and other health system data to deliver...Full timeShift work- ...team made up of 2–3 developers , a product manager, and 1 Quality Advocate . The Quality Advocate is not solely responsible for... ...Development leadership. At Ncontracts, we embrace an AI-first engineering culture . Our QA team actively uses agentic development tools...Full timeShift work
Ncontracts
Remote12 days ago
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to AI Quality Engineer. Be the first to apply!
Related searches
- ai engineer United States
- ai research engineer United States
- machine learning ai engineer United States
- senior ai engineer United States
- ai ml engineer United States
- ai prompt engineer United States
- ai engineer remote United States
- ai developer United States
- design quality engineer United States
- program quality engineer United States