AI QA Engineer

QA Engineer – Agentic ERP Platform

The QA Engineer is responsible for hands-on testing and quality assurance of Rimini Street's Agentic ERP Platform, ensuring that AI-driven features meet functional requirements, performance standards, and enterprise reliability expectations. This role executes test plans, builds automated test suites, and identifies defects across the platform's APIs, UI, and agent workflows.

Reporting to the QA Testing Lead, this engineer works closely with developers to validate new features, reproduce and diagnose issues, and maintain comprehensive regression coverage. The ideal candidate is detail-oriented, experienced in test automation, and comfortable working with AI-powered systems where outputs may be non-deterministic.

Essential Duties & Responsibilities

• Execute manual and automated test cases for platform features, including API endpoints, UI components, and agent workflows.
• Develop and maintain automated test suites using Python-based testing frameworks.
• Write and maintain integration tests that validate end-to-end platform functionality.
• Perform regression testing to ensure new changes do not degrade existing functionality.
• Validate AI agent outputs against defined accuracy thresholds and evaluation criteria established by the QA Testing Lead.

Platform-Specific Testing

• Validate tool invocations for schema compliance, idempotency, error handling, and permission gating.
• Test fallback behaviour across primary and secondary integration methods, ensuring graceful degradation when preferred paths are unavailable.
• Validate durable workflow behaviour: process restart survival, timer firing, and idempotent retry correctness.
• Verify audit log completeness, ensuring every agent action produces a structured audit record with required fields (session, event type, agent, policy version, before/after state).

Defect Management

• Identify, document, and track defects with clear reproduction steps and severity assessments.
• Work with developers to diagnose root causes and verify fixes.
• Perform exploratory testing to uncover edge cases and unexpected behaviors.
• Participate in defect triage and prioritization sessions.

Test Infrastructure

• Maintain and improve test automation infrastructure and CI/CD test pipelines.
• Manage test data and test environment configurations across containerized deployments.
• Build and maintain test fixtures and mock services for isolated testing.
• Monitor test execution results and investigate flaky or unreliable tests.

Quality Processes

• Contribute to test plan development and test case design.
• Participate in sprint planning and estimation for QA activities.
• Document testing processes and maintain knowledge base articles.
• Support release validation and sign-off activities.

Experience

• 3+ years of QA engineering experience, with hands-on test automation.
• Experience testing REST APIs and web applications.
• Familiarity with agile development processes and CI/CD workflows.
• Experience with at least one test automation framework (Pytest, Selenium, Playwright, or similar).
• Exposure to AI/ML system testing or non-deterministic output validation is a plus.
• Enterprise software or B2B platform experience preferred.

Technical Skills

Required

• Python for test scripting and automation.
• Test automation frameworks: Pytest, Selenium, or Playwright.
• API testing: Postman, REST Assured, or equivalent.
• API contract testing and schema validation (Pact or equivalent).
• Docker and containerized test environments.
• SQL fundamentals for test data validation (PostgreSQL preferred).
• Git version control and pull request workflows.
• Basic understanding of CI/CD pipelines (GitHub Actions, Jenkins).
• Defect tracking and test management tools (Jira, TestRail, or equivalent)

Preferred

• Experience testing Angular web applications (Angular 14+).
• Familiarity with performance testing tools (JMeter, Locust, k6).
• Observability tool literacy: OpenTelemetry traces, Jaeger, LangFuse, or Grafana for distributed trace inspection.
• Evaluation framework familiarity: LLM-as-judge, Promptfoo, DeepEval, or equivalent for AI output regression testing.
• Understanding of durable execution testing patterns (workflow replay, idempotency verification).
• Exposure to testing AI/ML outputs: evaluation metrics, statistical validation.

Skills & Competencies

• Detail-oriented; able to design thorough test cases that cover edge cases and boundary conditions.
• Analytical mindset; can diagnose complex issues across multiple system components.
• Collaborative; works effectively with developers, QA leads, and cross-functional teams.
• Proactive; identifies testing gaps and suggests improvements to processes and coverage.
• Clear communicator; writes precise bug reports and test documentation.
• Self-motivated and effective in a remote environment.
• Fluent in English (written and verbal).

Desired Qualifications

• Bachelor's degree in Computer Science, Software Engineering, or related field.
• ISTQB Foundation or similar QA certification.
• Experience in enterprise software companies or B2B SaaS platforms.
• Contributions to open-source testing tools or frameworks.

Location & Travel

Location: Hybrid - Selangor or Penang office.

Travel: Minimal; occasional travel for team meetings or training

Language: Fluent English required (written and verbal)

Why Rimini Street?

We are looking for talented, passionate people to help us build our future at Rimini Street. We hire only the best, the most extraordinary professionals and provide compensation, bonuses, and benefits to match the skills of our top-performing team members. Do you thrive in a fast-paced environment, enjoy growing together, and get excited about learning new skills? Are you looking for an opportunity to make a true impact as part of a team of extraordinary professionals? This is the place for you.

Our work is challenging and meaningful. We start and end each day with a sense of achievement and purpose guided by our core values, the Four Cs:

• Company: We dream big and innovate boldly.
• Colleagues: We work with extraordinary people who create a culture of mutual respect and collaboration.
• Clients: We relentlessly pursue solutions that help clients achieve their goals. Our unmatched client care is rooted in our passion for exceptional service.
• Community: We believe in leaving the world a better place than we found it. With the Rimini Street Foundation, we've made positive impacts in six continents for over 425 charities.

Accelerating Company Growth

• Nasdaq-listed under ticker symbol RMNI since October 2017
• Over 6,300+ signed contracts to date, including Fortune 500 and Global 100 companies
• Over 2,000 team members in 23 countries
• US and international recognition for industry leadership and philanthropic efforts.

Rimini Street is committed to creating a diverse and inclusive environment and is proud to be an Equal Employment Opportunity Employer. All qualified applicants will receive consideration for employment without regard to age, race, color, religion, national origin, sexual orientation, gender or gender identity, disability, protected veteran status, or any other characteristic protected by law.