Quality Assurance Test Lead

Position: QA Lead

Location: Fort Mill, SC

Experience: 10-15 years

About Incedo

Incedo Inc. is a high-growth Digital, Data and AI Transformation Specialist firm headquartered in New Jersey. We are a long-term strategy execution partner for Fortune 500 enterprises, operating at the intersection of business and technology across Banking & Payments, Wealth Management, Telecom, Hi-Tech, and Life Sciences. We are building Incedo 4.0 - an AI-native, execution-focused, founder-led organization designed for scale, speed, and long-term impact.

Incedo delivers ROI from AI @ Scale through the “Power of 3”:

• Deep domain expertise

• AI & Data capabilities

• Engineering & Operations excellence

About the Role

We are seeking an experienced and results-driven QA Automation Lead to join our team. The ideal

candidate will be responsible for leading automation testing initiatives, designing and maintaining scalable automation frameworks, and ensuring the delivery of high-quality web and mobile applications.

This role requires strong expertise in Java, Selenium, Playwright, TestNG, and Automation Testing frameworks, along with hands-on experience in AI model evaluation and Generative AI testing. The candidate should be able to assess LLM behaviour, evaluate prompts and responses, validate AI/ML outputs, and collaborate effectively with developers, business analysts, product owners, and stakeholders to drive quality across the software development lifecycle.

Key Responsibilities

• Lead and drive end-to-end automation testing efforts for web and mobile applications.
• Design, develop, maintain, and execute automated test scripts using Selenium with Java, TestNG Framework, and Playwright.
• Develop and maintain robust automation frameworks that support regression, functional, and integration testing.
• Create comprehensive test plans, test suites, test scenarios, and automated/manual test cases.
• Plan and execute AI/ML and Generative AI testing, including model evaluation, prompt testing, response validation, and behavior validation.
• Define and execute test scenarios for LLM-based applications, including accuracy, consistency, relevance, robustness, safety, and response quality.
• Evaluate AI model outputs against expected behavior and business requirements and document defects, anomalies, and model-quality findings.
• Create comprehensive test plans and test cases covering both traditional software functionality and AI/LLM behavior.
• Collaborate closely with Developers, Product Owners, Business Analysts, and QA teams to implement Shift-Left Testing practices.
• Participate actively in Agile ceremonies, including sprint planning, reviews, and retrospectives.
• Identify, document, track, and verify defects using JIRA or similar defect management tools.
• Leverage modern AI-powered testing and productivity tools to improve test case creation, automation development, defect analysis, documentation, and overall QA efficiency.
• Utilize Generative AI tools such as GitHub Copilot, ChatGPT, Microsoft Copilot, or similar platforms to accelerate test automation development and enhance testing practices while adhering to organizational standards and security guidelines.

Required Qualifications

• Bachelor’s degree in computer science, Information Technology, Engineering, or a related field.
• 10–15 years of overall experience in Software Quality Assurance and Test Automation.
• Strong understanding of Object-Oriented Programming (OOP) concepts.
• Hands-on experience with Java programming.
• Expertise in Selenium WebDriver automation.
• Strong experience with Playwright automation framework.
• Extensive experience with TestNG framework/Page Object Model (POM).
• Working knowledge of Microsoft SQL Server and database validation techniques.
• 2+ years of hands-on experience in AI model evaluation, Generative AI testing, or ML validation.
• Hands-on experience designing and executing test scenarios for LLM-based or Generative AI applications, including prompt/response validation and model output evaluation.
• Hands-on experience using AI-powered development, testing, or productivity tools such as GitHub Copilot, Microsoft Copilot, ChatGPT, Gemini, Cursor, or similar solutions to improve software testing and automation efficiency.
• Understanding of AI-assisted software development and testing best practices.
• Strong analytical, troubleshooting, and problem-solving capabilities.
• Excellent verbal and written communication skills.

Preferred Qualifications

• Experience with API Testing using tools/frameworks such as Postman, Rest Assured, or similar.
• Exposure to Mobile Application Testing (Android/iOS).
• Experience with Azure DevOps/TFS Integration.
• Familiarity with CI/CD tools and automated deployment pipelines.
• Experience with test reporting and quality metrics dashboards.
• Experience with AI evaluation frameworks, LLM observability/testing tools, or automated evaluation techniques.