Quality AI Engineer

Quality AI Engineer

Location : Remote ( Candidates must be available for an in-person interview or laptop pickup at one of the following locations: San Francisco, CA | Arlington, VA | Denver, CO | Chicago, IL | Boston, MA | New York, NY | Houston, TX | Miami, FL | Los Angeles, CA | Seattle, WA | Dallas, TX | Minneapolis, MN | Birmingham, MI | Irvine, CA)

Duration : 6 10 Months

Interview : In-person on any of the mentioned location

Note:

• Must be local with DL copy
• LinkedIn with location and photo (before 2023)

Position Overview

We are seeking a highly skilled QA Automation Engineer with strong experience in Playwright, Cypress, Python, PyTest, and JMeter to design, develop, and execute automated testing solutions for enterprise applications and AI-driven platforms.

The ideal candidate will have hands-on experience testing AI/ML and Generative AI applications , including LLM-based applications, chatbots, and intelligent agents. This role will collaborate closely with Product, Engineering, AI/ML, Data Science, and DevOps teams to improve quality, automation coverage, performance, and release confidence.

Required Skills

• 5+ years of experience in QA, Quality Engineering, or Automation Testing.
• Strong hands-on experience with Playwright and Cypress for UI and end-to-end automation.
• Strong programming experience with Python .
• Hands-on experience building automation frameworks using PyTest .
• Experience with JMeter for performance and load testing.
• Strong understanding of software testing methodologies, SDLC, and QA best practices.
• Experience testing REST APIs and microservices .
• Experience with Git, CI/CD pipelines, and DevOps practices .
• Strong problem-solving, debugging, and analytical skills.

AI / GenAI Experience

• Experience testing AI/ML applications, Generative AI platforms, or intelligent automation solutions .
• Understanding of LLMs and AI application testing methodologies .
• Experience evaluating AI-generated responses for accuracy, relevance, hallucinations, consistency, and safety.
• Familiarity with AI model evaluation metrics and testing frameworks.
• Experience with AI technologies/tools such as OpenAI, Azure OpenAI, Claude, Gemini, LangChain, CrewAI, AutoGen , or similar platforms.
• Knowledge of prompt engineering and AI workflow validation .
• Experience creating automated tests for AI-powered APIs and conversational applications .

Other Skills

• Experience with AWS, Azure, or GCP .
• Experience with Docker, Kubernetes, and containerized environments .
• Familiarity with Jira, Azure DevOps, or TestRail .
• Experience working in Agile/Scrum environments.
• Experience with observability, monitoring, and application performance tools.