AI Agent Developer and Tester

Kforce has a client in SLC that is seeking an AI/Agent Developer & Tester to design, build, and validate LangGraph-based agent workflows within the APEX platform. Summary: This role is highly execution-focused, with ownership of testing strategy, quality, and reliability across AI agent systems-from unit-level validation through end-to-end workflows and production monitoring. The AI Agent Developer and Tester will be responsible for ensuring agent accuracy, stability, and scalability in a non-deterministic LLM environment, while also driving comprehensive test coverage across backend workflows and UI experiences. Responsibilities: * Develop and implement AI agent workflows using LangGraph / LangChain * Own end-to-end testing strategy, including:

• Node-level unit testing (isolated components with mocked state)
• Graph-level integration testing (full workflow execution)
• Regression test suite design and maintenance
• Establish LLM testing frameworks, including mocking, stubbing, and deterministic validation strategies for non-deterministic systems
• Manage flaky test mitigation, applying controlled seeds, mocked outputs, or assertion thresholds
• Own regression discipline-ensure every defect is paired with a regression test before release
• Build and maintain E2E test coverage using Playwright across APEX portals, actively identifying and closing coverage gaps
• Implement test coverage reporting and enforce quality standards
• Validate structured outputs (JSON schema enforcement)
• Ensure production observability of agent behavior and performance
• Support and manage cloud infrastructure (GCP preferred) for the agent ecosystem
• Hands-on experience with LangGraph and/or LangChain
• Strong Python development skills, including pytest and pytest-asyncio
• Proven experience testing AI/LLM-driven systems, including: Unit testing with mocked state; End-to-end workflow validation
• Experience implementing LLM mocking strategies to control non-deterministic outputs
• Experience with Playwright or similar tools for E2E UI testing
• Familiarity with structured data validation (JSON schema)
• Experience working in cloud environments (GCP preferred, others acceptable)
• Experience with LangSmith, RAGAS, or similar evaluation tools
• Prompt versioning and lifecycle management
• Experience with adversarial/red-team testing of LLM systems
• Experience with token usage and cost instrumentation
• Experience with document processing or OCR pipelines
• Background in financial services or regulated environments

• GCP (Cloud Run)
• Terraform
• Azure DevOps pipelines