Automation Engineer II

Automation Engineer II

• 10+ years of proven experience in the software development industry, participating in a team environment. • 5+ years of experience in UI automation, such as Selenium WebDriver. • 5+ years of proven experience in testing web services such as RESTful API. • 3+ years of proven experience in performance testing using tools such as Apache JMeter. • 3+ years of working in a cloud-based development and production environment such as AWS. • 3+ years of proven experience with programming languages such Java/C#. Familiarity with Python is a plus. Seeking candidates with solid coding proficiency in Java or C#. • Proficient with both SQL and NoSQL databases. • Experienced working in Continuous Integration/Continuous Deployment (CI/CD) environments. • Having AWS/Cloud technologies and AI/ML powered applications knowledge are big plus. • Experience with AI/ML-powered applications, Generative AI, and Agentic AI systems is highly desirable. • Understanding of AI testing concepts, including prompt validation, model output verification, hallucination detection, bias testing, guardrail validation, and AI quality evaluation. • Familiarity with LLMs (Large Language Models), AI agents, retrieval-augmented generation (RAG), and AI-assisted workflows is a strong plus. • Ability to develop QA strategies and test approaches for AI-driven and autonomous agent-based applications.

Key Responsibilities:

• Design and execute functional and automation testing for applications. • Develop and execute automated test scripts using testing tools; document and summarize results. • Build, update, and maintain UI automation test cases using Selenium with C# (flexibility to learn C# if not already proficient). • Perform manual API testing using tools such as Postman or Insomnia. • Perform API automation testing using industry-standard frameworks and tools. • Participate actively in Agile ceremonies (daily stand-ups, sprint planning, retrospectives). • Collaborate closely with Development and Business teams to ensure project success. • Quickly adapt to new technologies and tools (e.g., Amazon Web Services). • Work with SQL and NoSQL databases for backend validation and robust data testing. • Create comprehensive end-to-end test plans, test specifications, and test case templates. • Develop and document detailed test plans and test cases. • Facilitate defect tracking, reporting, and resolution management. • Review requirements (Functional Specifications, User Stories, Change Requests) to ensure full coverage. • Identify and manage test data; prepare automation-ready test cases. • Conduct system testing, regression testing, and support User Acceptance Testing (UAT). • Apply test case design techniques (e.g., boundary value analysis, equivalence partitioning). • Implement Shift Left Testing practices by participating in early requirement and design reviews, writing acceptance criteria up front, and integrating automated tests in CI/CD pipelines. • Be actively involved in the deployment process, collaborating with DevOps and engineering teams to ensure smooth releases. • Design and execute test strategies for AI/ML and Agentic AI applications, including: o Validating AI-generated outputs for accuracy, consistency, and reliability. o Testing AI agents for workflow execution, decision-making, and multi-step task completion. o Evaluating AI model behavior, prompt responses, hallucinations, and edge cases. o Verifying AI safety controls, guardrails, security, and compliance requirements. o Creating test datasets and benchmarks for AI quality assessment. o Monitoring and reporting AI model performance, accuracy, and user experience metrics.

Nice to Have:

• Experience with application monitoring tools (e.g., New Relic, Datadog, CloudWatch). • Working knowledge of Python for scripting and automation. • Familiarity with AWS services for cloud-based testing and deployment. • Hands-on experience with performance and load testing using JMeter • Experience with testing AI/ML-powered applications, including validating model outputs, creating test datasets, evaluating model performance.