Test Lead

Pinnacle Talent Placement

Worcester, MA

AI Test Lead

Status: Must be legally eligible to work where you live.

Resume must be in English

Location : Remote, some travel to Atlanta, GA

Engagement : Contract, Long Term, Full Time

About the Role

We are looking for an AI Test Lead with 10+ years of overall testing experience, including 2+ years testing AI/ML systems, to own the quality strategy for our AI-powered applications and models. You will define how we test AI systems end-to-end, lead testing execution using modern AI tools, and set the standard for reliable, trustworthy AI delivery.

Key Responsibilities

Define and own the AI testing strategy for applications and models.
Lead AI application testing across functional, integration, and user-facing behavior.
Perform model testing and validation (accuracy, robustness, bias, drift, hallucination, and performance).
Leverage AI-powered testing tools to design, automate, and accelerate test coverage.
Establish Agentic test frameworks, quality gates, and acceptance criteria for AI features.
Collaborate with data science, engineering, and product teams to embed quality early.
Report on quality metrics, risks, and readiness for release.

Required Skills & Experience

10+ years in software QA/testing, with 2+ years hands-on testing AI/ML applications and models.
Hands-on with AI-based testing tools and test automation.
Ability to create and drive AI testing strategy from the ground up.
Strong understanding of model evaluation metrics and AI failure modes.
Proven experience leading QA teams and mentoring testers.
Experience testing LLM / GenAI applications (prompt testing, RAG evaluation).
Familiarity with CI/CD and MLOps pipelines.

Nice to have:

LangFuse, Langsmith, Test Rigor of Functionize experience

Apply

Vacancy posted 1 day ago

Similar jobs that could be interesting for youBased on the Test Lead in Worcester, MA vacancy

Lab Supervisor: POC Testing & Cross-Campus Lead
...UMass Memorial Health is seeking a Laboratory Supervisor to oversee day-to-day staff, testing, and workflow across laboratories. You will ensure results accuracy, maintain CAP and regulatory standards, and provide advanced technical support to staff while coordinating...
Suggested
HAHHH
Worcester, MA
2 days ago
Oracle Fusion & Integration Testing Lead
...Job Title: Oracle Fusion & Integration Testing Lead Location: Remote Job Duration: Full-time Job Description: As our Oracle Fusion & Integration Testing Lead, you’ll own the testing strategy for Project Premiere, our global Oracle transformation across ERP...
Suggested
Full time
Remote work
Vantage Point Consulting Inc.
Worcester, MA
1 day ago
Lab Supervisor: Point-of-Care Testing Lead
UMass Memorial Health is seeking a Laboratory Supervisor to oversee daily operations, staff supervision, and QA programs across campus laboratories. The role requires a BC/MLS with national certification and leadership experience, ensuring compliance with CAP and Joint...
Suggested
Relocation package
Day shift
C100 UMass Memorial Medical Center, Inc.
Worcester, MA
23 hours ago
Head of QA and Test
About the role As a Test Manager, you will make an impact by leading end-to-end quality engineering and testing initiatives for strategic insurance technology programs. You will be a valued member of the Quality Engineering and Assurance team, working closely with client...
Suggested
Temporary work
Visa sponsorship
Cognizant
Worcester, MA
4 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Test Lead. Be the first to apply!