Automation Test Architect - Agentic AI

We are seeking a Test Automation Architect to define and lead an enterprise-grade test automation strategy across UI, API, data, and platform layers—while evolving our quality engineering capabilities to support Agentic AI / LLM-based systems (probabilistic outputs, confidence-based validation, hallucination detection, and model evaluation). This role will architect scalable automation frameworks, drive engineering excellence, and accelerate delivery through highly productive, maintainable automation programs. Key Responsibilities Automation Strategy & Architecture Define and own the test automation architecture and multi-layer strategy (UI/API/contract/integration/e2e), aligned to product risk and release cadence. Establish automation standards, patterns, and reference implementations (framework structure, page-object/screenplay usage, test data patterns, reporting). Drive shift-left testing practices (contract testing, component testing, pre-merge validation) and reduce reliance on fragile end-to-end tests. Agentic AI / LLM Test Model Alignment Design and implement testing approaches for non-deterministic AI outputs, including: confidence/threshold-based assertions semantic similarity checks and rubric-based validation goldens/benchmark datasets and evaluation harnesses hallucination and groundedness testing strategies regression detection across model/prompt/version changes Define AI test coverage across accuracy, robustness, bias/safety (as applicable), performance, and reliability. Partner with ML/AI, Product, and Engineering to establish acceptance criteria for AI capabilities and measurable quality gates. High-Productivity Automation Programs Build and scale high-throughput automation projects that are reliable, fast, and easy to extend. Improve test execution time, reduce flaky tests, and optimize CI/CD pipelines (parallelization, test selection, retries with guardrails). Create actionable, engineering-friendly quality dashboards (pass/fail trends, flake rate, coverage, defect escape metrics). Tooling, CI/CD, and Quality Gates Integrate automation into CI/CD with quality gates (PR checks, nightly suites, release readiness criteria). Standardize observability for testing: logs, tracing hooks, screenshots/video, artifacts, and test run diagnostics. Define practices for test environments, data management, service virtualization, and dependency control. Technical Leadership & Mentorship Serve as the QE technical authority: coach engineers on automation design, code quality, and best practices. Lead design reviews for test architecture and automation PRs; enforce coding standards and maintainability. Influence roadmap and delivery by translating risk into clear testing priorities. Required Qualifications 8–12+ years in Quality Engineering / Test Automation with architect-level ownership of frameworks and strategy. Proven experience designing scalable automation across: API testing (contract + integration) UI automation with stable locator strategies (test IDs, accessibility roles) Test data strategy and environment reliability Strong programming skills in at least one modern language (e.g., Java, Python, TypeScript/JavaScript, C#). Expertise with automation tools/frameworks (examples): UI: Playwright / Cypress / Selenium API: REST-assured / SuperTest / Postman/Newman / pytest CI: GitHub Actions / Jenkins / Azure DevOps Demonstrated ability to reduce flakiness and improve execution speed through engineering solutions. Preferred Qualifications (Strongly Desired) Experience testing LLM/Agentic AI features or other probabilistic systems, including: evaluation harnesses, benchmark datasets, semantic scoring, and drift detection prompt/version regression strategies and structured evaluation reports Familiarity with modern quality approaches: test pyramid, contract testing, consumer-driven contracts, risk-based testing, shift-left. Knowledge of security/performance testing integration (baseline understanding; hands-on is a plus). Experience building reusable automation libraries for multiple teams/products. Our Interview Practices To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to ensure we’re getting to know you—not a digital assistant. To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process. Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process. Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process. Compensation: $89,600.00 - $157,000.00 USD This role is eligible for Bonus. Compensation range listed is based on primary location of the position. Actual base salary offer is influenced by a wide array of factors including but not limited to skills, experience and actual hiring location. Your recruiter can share more information about the specific offer for the job location during the hiring process. Additional Information: Wolters Kluwer offers a wide variety of competitive benefits and programs to help meet your needs and balance your work and personal life, including but not limited to: Medical, Dental, & Vision Plans, 401(k), FSA/HSA, Commuter Benefits, Tuition Assistance Plan, Vacation and Sick Time, and Paid Parental Leave. Full details of our benefits are available upon request. If making a difference matters to you, then you matter to us. Join us, at Wolters Kluwer, and be part of a dynamic global technology company that makes a difference every day. We’re innovators with impact. We provide expert software and information solutions that the world’s leading professionals rely on, in the moments that matter most. You’ll make a real difference in the lives of millions of people. Together with our customers, we help to advance sustainability, health, justice, prosperity, and commerce around the world. You can thrive at Wolters Kluwer, where diversity is core to our collective strength and high performance. Be your unique self, share your creative ideas, do your best work, and take time to grow in our caring and inclusive culture where you can belong. Wolters Kluwer reported 2025 annual revenues of €6.1 billion. The group serves customers in over 180 countries, maintains operations in over 40 countries, and employs more than 21,000 people worldwide. Our customers work in industries that impact the lives of millions of people every single day. Our mission is to empower our professional customers with the AI-powered solutions they need to make critical decisions, achieve successful outcomes, and increase productivity. We deliver trusted, AI-powered expert solutions that combine deep domain knowledge, proprietary content, and advanced technology to provide expert-validated insights, automate workflows, and drive better outcomes. Today, nearly 70% of our digital revenues come from AI-powered solutions, reflecting our strategic focus on embedding AI into professional workflows and delivering measurable productivity gains. We are committed to helping professionals navigate complexity, improve the way they work, and solve critical challenges through cloud-based, AI-powered platforms that integrate seamlessly into their workflows and ecosystems. Our solutions are designed to deliver impact when it matters most, especially in high-stakes environments where accuracy and trust are essential. With a nearly 190-year legacy, Wolters Kluwer continues to evolve its portfolio of digital solutions and services to meet the changing needs of professionals worldwide, enabling them to work faster, make smarter decisions, and deliver better outcomes. For more information about our solutions and organization, visit or follow us on LinkedIn, Instagram and Facebook. Wolters Kluwer has a dedicated team of experienced talent acquisition professionals who are ready to hear what makes you tick and share how Wolters Kluwer can help you achieve your goals and ambitions. Here Rasi Fawaz shares some tips on what recruiters are looking for on a resume. As a global organization, we recognize that solving a wide range of complex problems requires diverse perspectives and innovative thinking. We know that bringing our best to our customers, communities, and other key stakeholders requires a highly engaged and talented workforce – one that represents the diversity of those we serve and the communities where we live and work. We aim to provide a welcoming environment and equitable opportunities for all employees regardless of background, nationality, race, ethnicity, gender, gender identity, age, sexual orientation, marital status, disability, or religion. This principle is ingrained in our company values and articulated in our Code of Business Ethics. At Wolters Kluwer, you’ll feel valued for your contributions, and you’ll know that your health, safety, and well-being are important to us. Our well-being benefits provide tools, programs, and resources to help our diverse employees feel healthy, happy, safe, and prosperous. Our global Together we thrive well-being program reflects our commitment to supporting colleagues across physical, mental, social, and financial well-being. That commitment has been recognized externally through Ragan’s Top Places to Work for Employee Wellbeing: Large Organization in 2024. More recently, Wolters Kluwer was recognized by Forbes as one of America’s Best Large Employers 2026 and America’s Best Employers for Engineers 2026. In 2025, Wolters Kluwer was also recognized by Forbes as one of America’s Best Employers for Women. To ensure we continue to drive innovation that enables us to develop products and services to best serve our customers, we cultivate a workplace culture rooted in mutual respect, bringing forward insights from a wide range of backgrounds, perspectives, and experiences. We are also committed to complying with laws requiring equal opportunity in hiring, promotion, and other employment decisions. All qualified applicants will receive consideration without regard to race, color, religion, sex (including pregnancy, gender identity, transgender status, and sexual orientation), national origin, disability, age, genetic information, veteran status, or any other characteristic protected by applicable law, and we do not tolerate discrimination on any of these bases. GDPR Wolters Kluwer (“we” or “us”) wants to inform you about the ways we process your personal information. In this Privacy & Cookie Notice we explain what personal information we collect, use and disclose. You'll find details related to GDPR and other data privacy policies on our Careers Privacy and Cookies page.