AI QA Engineer

Job Title: AI QA Engineer
Location: Rosemead, CA 91770 (Hybrid)
Duration: 06 Months


Must Have Skills:
• QA/test planning, exploratory testing, and UAT experience
• AI/ML fundamentals and evaluation metrics literacy
• Data analysis and model behavior reasoning
• Ability to test GenAI prompts and outputs (preferred)
• Strong domain knowledge and user-centric mindset
• Clear documentation and communication skills


Nice to Have Skills:
• 3-7 years in QA, testing, or quality roles
• Experience with AI/data-driven products preferred
• ISTQB, AI/Data, or Responsible AI training is a plus


Detailed Job Description:
Role Summary
• Ensures AI products behave accurately, safely, fairly, and reliably across real-world scenarios. Partners with Product, Engineering, and SMEs to validate model behavior, data quality, and trust risks across the AI lifecycle.


Key Responsibilities
• Test AI outputs for accuracy, consistency, edge cases, and failure modes
• Validate training and inference data quality; identify bias and data leakage risks
• Define and execute model evaluation and regression tests
• Reuse and expand the Golden Dataset for regression testing across builds, prompt/model changes, and releases.
• Test AI behavior within end-to-end user workflows and integrations
• Support Responsible AI testing (fairness, explainability, compliance)
• Monitor model performance post-release (drift, feedback, regressions)
• Test automation coverage
• Communicate risks and test findings clearly to stakeholders
• Track & report quality metrics such as accuracy, precision (and recall where applicable), clearly highlighting missing, incorrect, or hallucinated content.


Minimum Years of Experience:
• 3-7 years