Quality Analyst

QA Engineer – Insights (AI/ML)

Location: Remote (US) Department: Insights (AI/ML) Reports to: Engineering Manager

About the Role

Irth is building a new AI-driven threat and risk management platform for pipeline asset integrity. The platform brings together three capabilities that have historically been separate at Irth:

• A governed, cross-product data platform built on Databricks and Azure
• An AI-powered ingestion layer that normalizes, repairs, and enriches customer data without services-heavy onboarding
• A reusable analytical layer that runs industry-standard, Irth-developed, and customer-built risk models against the data

We are seeking a mid-level QA Engineer to own quality across the platform end to end, including the user interface, APIs and platform services, and the underlying analytics and model layer.

This role goes beyond traditional application testing. You will validate that data survives ingestion and transformation intact, risk-model outputs are accurate and reproducible, and the evidence trail behind those outputs can withstand regulatory audit. Operators use these outputs to prioritize excavation and repair work, so a silently incorrect result can be far more consequential than a visibly broken interface.

You will own regression coverage and test automation while helping build quality into the delivery pipeline from the beginning rather than inspecting it at the end.

Key Responsibilities

• End-to-End Test Ownership — Primary Responsibility
• Own the end-to-end test strategy across the user interface, APIs, and analytics layers, defining the appropriate coverage at each level.
• Design, build, and maintain automated test suites covering UI, API contracts and integrations, data validation, and model validation.
• Build and maintain regression coverage that runs on every change and provides a reliable signal for release readiness.
• Create and manage test data, including realistic messy inputs that reflect the quality and variability of actual customer data.
• Perform exploratory testing on new functionality to identify failures that scripted tests may not detect.
• Data & Model Validation
• Validate data accuracy throughout ingestion and transformation, including row-level and field-level reconciliation, schema conformance, and completeness checks across Bronze, Silver, and Gold layers.
• Validate model outputs against expected results, known baselines, and golden datasets, including verification that identical inputs produce consistent and reproducible outputs.
• Test edge cases throughout the risk-calculation path, including missing attributes, boundary values, unusual segment geometry, and conflicting source records.
• Validate geospatial correctness, including alignment of results to pipeline centerline geometry and consequence-area assignment.
• Verify AI-assisted extraction and gap-filling workflows, ensuring low-confidence outputs are routed for human review rather than silently accepted.
• CI/CD & Test Automation Infrastructure
• Integrate automated test suites into CI/CD pipelines with appropriate quality gates at each environment promotion.
• Own test-environment configuration, data seeding, and teardown, including infrastructure-as-code contributions where appropriate.
• Build and maintain the test automation framework, keeping suites fast, stable, maintainable, and trustworthy. Treat flaky tests as defects.
• Report quality signals to the team, including coverage, pass rates, defect escape rate, and quality trends over time.
• Release Readiness & Non-Functional Testing
• Coordinate release testing and sign-off, including assessing what changed and what functionality could plausibly be affected.
• Support performance and load testing of APIs and analytical workloads ahead of release milestones.
• Support security testing activities, including access-control verification and tenant-isolation testing.
• Verify upgrade and migration paths, including data migrations from legacy formats and systems.
• Audit-Defensible Test Evidence
• Produce and retain test evidence sufficient to support regulatory and audit review of risk-model outputs.
• Maintain traceability from requirements through test cases and execution results.
• Document validation approaches, test results, and known limitations for each model release.
• Collaboration & Quality Culture
• Work with Product to translate requirements into clear, concrete, and testable acceptance criteria before development begins.
• Partner with engineers on testability, unit and integration coverage, and defect triage.
• Advocate for quality during design reviews and identify risks early rather than waiting until the end of a sprint.
• Maintain clear defect-management practices, including reproducible steps, severity assessment, impact analysis, and verification of fixes.