Quality Engineer

Work at Home

Full time

JR-916886

Your Future Evolves Here

Evolent partners with health plans and providers to achieve better outcomes for people with most complex and costly health conditions. Working across specialties and primary care, we seek to connect the pieces of fragmented health care system and ensure people get the same level of care and compassion we would want for our loved ones.

Evolent employees enjoy work/life balance, the flexibility to suit their work to their lives, and autonomy they need to get things done. We believe that people do their best work when they're supported to live their best lives, and when they feel welcome to bring their whole selves to work. That's one reason why diversity and inclusion are core to our business.

Join Evolent for the mission. Stay for the culture.

What You’ll Be Doing:

Position Summary

The Quality Engineer develops and executes test plans, performs exploratory and automated testing, and builds and maintains automated tests for the Evolent Platform web user experience. Working in an Agile, globally dispersed team, this role partners with developers, business analysts, and stakeholders to understand requirements and expected agent behavior, surface quality issues early, and report clearly on quality status.

This individual works with minimal supervision on assigned features, actively learning and adopting the testing methods, automation frameworks, and patterns defined for the role and by the team. The role is expected to use approved AI-assisted tools to accelerate testing and improve quality while remaining fully accountable for every delivered result.

AI-Assisted Engineering Expectations

Evolent engineers are expected to effectively leverage approved AI-assisted development and productivity tools (e.g., code assistants, copilots, automation agents, knowledge assistants, and analytics tools) to improve quality, efficiency, and innovation while maintaining accountability for all delivered work. These expectations span the full engineering lifecycle, not only code generation, including iterating with AI on test design and solution design, evaluating alternatives, and producing design artifacts. Engineers remain responsible for validating outputs, ensuring solution quality, protecting sensitive information, and complying with security, privacy, regulatory, and Responsible AI standards.

Key Responsibilities

Test Planning and Execution

• Develop, document, and execute test plans and test cases for assigned features.

• Perform exploratory and automated testing to ensure software functionality, reliability, and performance.

• Conduct regression testing to ensure new code does not adversely impact existing functionality.

• Collaborate with development teams to understand requirements, expected agent behavior, and human-in-the-loop flows, and identify potential quality issues early.

• Participate in code reviews, design discussions, and sprint planning, and contribute to the design and implementation of enhancements to the team’s test systems.

Test Automation and CI/CD

• Create and maintain Playwright end-to-end tests and Vitest component tests using the team’s established frameworks and patterns.

• Execute and maintain contract tests and API tests using the team’s established tooling.

• Test streaming, loading, and error states, including slow and failed agent responses.

• Execute automated test suites in GitHub Actions, evaluate results, and fix failing scripts to maintain a high pass rate.

• Create and maintain synthetic test data.

• Follow and help reinforce the team’s testing processes and automation standards, holding peers accountable constructively and making sure the team understands the risks of exceptions.

Accessibility Testing

• Run automated axe-core accessibility checks and manual screen reader tests (JAWS, NVDA, VoiceOver) against WCAG 2.1/2.2 AA.

Defect Management and Quality Reporting

• Identify, isolate, document, and track defects, working closely with developers to ensure timely resolution.

• Analyze test and evaluation results, generate quality metrics, and report on software quality status to the team and stakeholders.

• Present work in Agile ceremonies, participate in knowledge-sharing sessions, and seek feedback to continuously improve skills.

AI-Enabled Quality Engineering

• Utilize approved AI-assisted engineering tools to accelerate test case design, test script development, test data generation, documentation, and troubleshooting.

• Evaluate and validate AI-generated tests, scripts, and analysis for correctness, security, maintainability, compliance, and business appropriateness before use.

• Learn and adopt the prompts, skills, agents, and AI-assisted testing patterns defined by the team and share improvements that benefit the team.

• Identify opportunities to improve testing productivity and quality through AI-enabled workflows and automation.

Governance and Compliance

• Follow enterprise policies governing AI usage, intellectual property protection, security, privacy, PHI handling, and Responsible AI practices.

• Ensure no confidential, regulated, or protected information (including PHI in test data) is used with unapproved AI services or tooling; use synthetic data for testing.

• Participate in peer reviews and validation processes for AI-assisted solutions.

Required Qualifications

• 3+ years of experience in software quality assurance or testing, including hands-on UI test automation.

• Experience with Playwright or similar end-to-end automation tools, preferably using TypeScript or JavaScript.

• Working knowledge of JavaScript or TypeScript for test automation.

• Familiarity with component testing frameworks such as Vitest or Jest.

• Familiarity with API testing tools (e.g., Postman) and REST APIs.

• Familiarity with CI/CD tools (e.g., GitHub Actions) and version control systems such as Git.

• Familiarity with test management and defect tracking tools (e.g., JIRA, ALM).

• Proficiency in software testing methodologies, including functional, regression, and performance testing.

• Knowledge of Agile methodologies and experience working in an Agile development environment.

• Strong analytical skills and attention to detail, with a focus on troubleshooting and problem-solving.

• Excellent communication skills, openness to feedback, and the ability to collaborate effectively with cross-functional teams.

Preferred Qualifications

• Exposure to testing AI or LLM output using test datasets and scoring rubrics.

• Familiarity with testing React applications and streaming user interfaces.

• Familiarity with contract testing and basic load testing.

• Working knowledge of accessibility testing, including axe-core and screen readers (JAWS, NVDA, or VoiceOver).

• Exposure to AI-assisted testing tools such as GitHub Copilot or Playwright MCP.

• Familiarity with FHIR R4.

• Experience in healthcare or other regulated environments handling PHI.

Education

• Bachelor’s degree in Computer Science, Software Engineering, Information Technology, or a related field, or equivalent experience.

#EvolentAi

To ensure a secure hiring process we have implemented several identity verification steps, including submission of a government issued photo ID. We conduct identity verification during interviews, and final interviews may require onsite attendance. All candidates must complete a comprehensive background check, in-person I-9 verification, and may be subject to drug screening prior to employment. The use of artificial intelligence tools during interviews is prohibited and monitored. Misrepresentation will result in immediate disqualification from consideration.

Technical Requirements:

We require that all employees have the following technical capability at their home: High speed internet over 10 Mbps and, specifically for all call center employees, the ability to plug in directly to the home internet router.

Evolent is an equal opportunity employer and considers all qualified applicants equally without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, veteran status, or disability status. If you need reasonable accommodation to access the information provided on this website, please contact View email address on click.appcast.io for further assistance.

The expected base salary/wage range for this position is $91,000-96,000. As part of our total compensation package, Evolent is proud to offer comprehensive benefits (including health insurance benefits) to qualifying employees. All compensation determinations are based on the skills and experience required for the position and commensurate with experience of selected individuals, which may vary above and below the stated amounts.

Don't see the dream job you are looking for? Drop off your contact information and resume and we will reach out to you if we find the perfect fit!