AI Quality Engineer

AI Quality EngineerSoftware Guidance & Assistance, Inc., (SGA), is searching for an AI Quality Engineer for a contract assignment with one of our premier financial services clients in Irving, TX.ResponsibilitiesDesign, develop, and implement comprehensive quality assurance strategies for our artificial intelligence applications and AI agents.Responsible for creating robust test plans and procedures to validate the performance, functionality, accuracy, and resilience of AI models and systems.Play a critical role in ensuring our AI solutions are reliable, robust, and deliver intended value in production environments.Quality Assurance Strategy & Standards: Design and develop overall quality assurance plans, procedures, and standards specifically tailored for AI/Machine Learning models and systems, including Generative AI solutions and autonomous AI agents.Test Strategy & Plan Development: Create detailed test strategies and test plans to thoroughly evaluate the robustness, functionality, performance, and ethical behavior of AI agents and models. This includes defining test objectives, scope, resources, and schedules.Test Case Design & Execution: Develop, implement, and execute a wide range of test cases (functional, non-functional, performance, stress, regression, adversarial testing) to validate AI model outputs, decision-making processes, and overall system behavior.AI Robustness & Functionality Testing: Focus on verifying AI agent robustness against various inputs, edge cases, and unexpected scenarios, ensuring they consistently deliver accurate and reliable results while adhering to design specifications.Performance & Scalability Testing: Design and conduct tests to measure AI model inference speed, resource utilization, and scalability under different load conditions.Data Quality & Model Bias Assessment: Contribute to ensuring data quality for training and validation, and work with data scientists to design tests for identifying and mitigating model bias and fairness issues.Automation Development: Develop and maintain automated testing frameworks and scripts for AI/ML pipelines, model validation, and system integration.Defect Management: Identify, document, track, and prioritize defects and inconsistencies in AI model behavior and system functionality, working closely with development teams for timely resolution.Metric Definition & Reporting: Define key quality metrics (e.g., accuracy, precision, recall, F1-score, latency, reliability) and provide regular status updates and quality reports to stakeholders.Collaboration & Communication: Collaborate effectively with data scientists, machine learning engineers, software developers, product managers, and business stakeholders to understand requirements and integrate quality throughout the AI development lifecycle.Continuous Improvement: Stay abreast of the latest advancements in AI testing methodologies, tools, and best practices, and actively contribute to the continuous improvement of our AI development and deployment processes.Required SkillsBachelor's or Master's degree in Computer Science, Software Engineering, Quality Assurance, or a related technical field.3+ years of experience in Quality Assurance or Software Testing, with at least 1-2 years specifically focused on AI/Machine Learning systems.Strong understanding of AI/ML concepts, including supervised, unsupervised, and reinforcement learning, as well as knowledge of different AI model types (e.g., Generative AI, predictive models, classification models).Experience in designing and developing test strategies, test plans, and test cases for complex software systems, particularly those involving AI components.Proficiency in at least one programming language (Python strongly preferred) for test automation and data analysis.Familiarity with AI/ML frameworks (e.g., TensorFlow, PyTorch) and relevant testing libraries.Experience with various testing methodologies and tools (e.g., unit testing, integration testing, API testing, performance testing tools).Excellent analytical skills and attention to detail, with the ability to identify subtle anomalies in AI behavior.Strong communication and collaboration skills, with the ability to articulate technical issues clearly.