Test & Evaluation AI SME
$170k - $200kSystems Planning and Analysis, Inc.
Overview Systems Planning and Analysis, Inc. (SPA) delivers high-impact, technical solutions to complex national security issues. With over 50 years of business expertise and consistent growth, we are known for continuous innovation for our government customers, in both the US and abroad. Our exceptionally talented team is highly collaborative in spirit and practice, producing Results that Matter. Come work with the best! We offer opportunity, unique challenges, and clear-sighted commitment to the mission. SPA: Objective. Responsive. Trusted. The Joint, Office of the Secretary of Defense, Interagency Division (JOID) provides expert support services to a range of customers spanning across the Department of Defense, Federal Civilian, and international markets. JOID provides a diverse portfolio of analytical and programmatic capabilities to help our customers make informed decisions on their most challenging issues. The Critical Infrastructure Group (CIG) within JOID provides technical, programmatic and analytical support to customers with missions in national security to include the Department of Homeland Security (DHS), Health and Human Services (HHS), and the Federal Aviation Administration (FAA). The Group brings technical and operational expertise in fields of countering weapons of mass destruction, critical infrastructure protection, emergency response, nuclear non-proliferation, weapons deployment programs, and transportation safety. In addition to management and advisory services, CIG staff collaborate closely with our clients to provide high-quality modeling, simulation, software, and analytical products to meet the most demanding government standards and timelines. We have a near-term need for a Testing and Evaluation Artificial Intelligence SME to provide hybrid support for DHS customers located in Washington, DC. Responsibilities The T&E Analyst will review and comment on revisions of program T&E documents and provide the Government advice. Support multiple DHS acquisition programs, especially in capabilities that involve AI. Provide AI T&E expertise to assist the Government with ensuring that DHS acquisition programs generate suitable, executable T&E approaches that adequately assess the effectiveness, suitability, reliability, robustness, security, explainability, and operational risks of AI technologies embedded in technical solutions. Provide support, guidance, recommendations, and mentoring to all Deputy Directors regarding AI T&E and the development of DHS standards for evaluating embedded AI technologies, including reviews of program TEMPs, Operational Assessment/Test Plans, Operational Assessment/Test Reports, and Letters of Assessment. Provide support to the instructional advisor regarding the inclusion of AI T&E within the T&E curriculum. Provide, as directed, focused training on AI T&E. Research Government and industry acquisition and T&E sources and analyze results for applicable AI evaluation standards, tools, metrics, plans, and procedures. Qualifications Required Advanced Degree or equivalent credits/certifications. 10+ years of relevant experience. Active Secret clearance . Demonstrated skills and abilities to develop and implement strategies, standards, and methods to integrate AI T&E activities-including, but not limited to, data quality and representativeness assessments, AI performance and robustness testing, bias assessments, human-AI interaction evaluations, adversarial testing, and post-deployment monitoring-in support of acquisition programs, including, but not limited to, aviation, detection equipment, cybersecurity, information technology, decision-support systems, and autonomous systems. Desired DHS T&E Level 3 equivalent certification. Pay Range Information At SPA, we strive to deliver a robust total compensation package that will attract and retain top talent. Elements of the compensation package include competitive base pay and variable compensation opportunities. SPA provides eligible employees with an opportunity to enroll in a variety of benefit programs, generally including health insurance, flexible spending accounts, health savings accounts, retirement savings plans, life and disability insurance programs, and a number of programs that provide for both paid and unpaid time away from work. The specific programs and options available to any given employee may vary depending on eligibility factors such as geographic location, date of hire, etc. Please note that the salary information shown below is a general guideline only. Salaries are commensurate with experience and qualifications, as well as market and business considerations. Washington, DC, Pay Transparency Salary range: USD $170,000.00/Yr. - USD $200,000.00/Yr. #J-18808-Ljbffr Systems Planning and Analysis, Inc.
- Systems Planning and Analysis, Inc. seeks a Testing and Evaluation AI Subject Matter Expert to provide hybrid support for DHS customers in Washington, DC. The role focuses on integrating AI T&E into DHS acquisition programs, developing robust evaluation methods, and mentoring...SuggestedTemporary work
- ...We are seeking an experienced Engineer, AI – AI Evaluation & Model Risk Lead to lead how AI models are evaluated, cleared, monitored, and... ...ongoing model evaluation, model cards, evaluation harnesses, and test suites to ensure AI models are measured, documented, and...SuggestedFull timeWork experience placement
- ...We are looking for a GenAI/LLM Test Engineer with strong experience in AI/ML product testing, Python, AI evaluation frameworks, SQL, and test automation . The role will focus on testing LLM-based applications, RAG systems, AI agents, prompts, model outputs, and data...SuggestedFull time
- ...Services team, you will be the SME on deploying and operating... ...automation.Apply responsible AI-assisted development workflows... ...engineers, and raise standards across testing, continuous integration/... ...gatesPartner with internal teams and evaluate external vendors when it...Suggested
$35 - $42 per hour
Test & Evaluation Lab Tech - Civil, Electrical, & Mechanical Engineering A client of Innova Solutions is immediately hiring for an Test & Evaluation... ...laws. By applying for this job, you agree to receive calls, AI-generated calls, text messages, or emails from Innova...SuggestedHourly payFull timeContract workTemporary workWork experience placementImmediate startWorldwideFlexible hours- TekWissen Group is seeking a Lead II - Software Testing in Bellevue, WA for a 6-month hybrid assignment. You will design and execute test plans for AI/ML features, build automated test suites, and evaluate model outputs for accuracy, bias, and robustness. Collaboration...
- ...Tech Mirrors in Bellevue, WA, seeks an AI/ML Lead to drive development and governance of AI models... .... You will own model risk clearance, ongoing evaluation, and the creation of model cards, ensuring clear records of testing and clearance. responsible for building evaluation...
- ...About VTI Aerospace VTI Aerospace builds AI-powered perception and pilot assist... ...Role As a Senior Software Engineer – Evaluation, you will design and implement systems that... ...for automated benchmarking and regression testing Design and maintain systems for monitoring...
- ...Job Description Job Description Shield AI is a venture-backed defense-tech company... ...Manager of Propulsion to lead propulsion test development and execution for the X-BAT... ...transient characterization, thermal performance evaluation, and operability validation. Drive...Full timeTemporary workPart timeWorldwide
$142.3k - $263.3k
...Software Engineer In Test -AI/ML Imagine what we could do together. At Apple, new ideas have a way of becoming extraordinary products, services, and customer experiences very quickly. Bring passion and dedication to your job, and there's no telling what we could accomplish...Relocation- ...Amazon.com Services LLC's AGI Responsible AI team seeks an Applied Scientist III to advance evaluation of frontier models, focusing on safety, fairness, and trust. You will design benchmarks, metrics, and scalable evaluation systems for large language models and generative...
- ...Bellevue, WA is seeking a QA Engineer specialized in AI products for a 6-month contract. The role is hybrid... ...-person collaboration. You will design and execute tests for AI/ML features, build automated suites, and evaluate model outputs for accuracy, bias, and safety, while...Contract workLocal area
$110k - $220k
...Decision Intelligence team, part of Applied AI, develops advanced AI and mathematical... ...business environments. What you'll do:Develop, test, and deploy advanced data science... ...and algorithms for accuracy and scalability.Evaluate model performance using rigorous validation...Full timeTemporary workPart time$161k - $221k
Secure Every Identity, from AI to HumanIdentity is the key to unlocking... ...subject matter expert (SME) for solution engineering and... ...space.Lead technical efforts to evaluate, design, and implement new... ...candidate may be subject to drug testing to the extent required by U.S....Local areaWorldwideFlexible hours- ...ITAR exposure, S/SAR Clearance Shield AI is a venture-backed defense-tech company... ...Engineering Manager to lead the X-BAT Hardware Test Software team. This team builds and... ...data collection, and automated pass/fail evaluation. Our systems interact with custom fixtures...Full timeTemporary workPart timeWorldwide
$198.36k - $416.1k
...Senior Machine Learning Engineer technical leader to help build an AI Risk & Compliance Intelligence Platform and an Intelligent Data... ...in Seattle, WA combines architecture and execution across LLM evaluation, synthetic data generation, and modular agent systems that support...Temporary workShift work$204.51k - $269.94k
...strategy and future growth.As Director of AI, you will lead the team that builds and... ...outcome measurement — and make them reliable, evaluated, and economical at scale.This role sits... ...systems, APIs, data pipelines, testing, and production operations.Genuinely hands...Full timePart timeWork experience placementWork at officeLocal areaRemote workWork from homeFlexible hoursShift work3 days per week1 day per week- We are:Accenture’s Global Responsible AI team within the Global Data & AI Practice. AI... ...and responsible practices.Detecting, evaluating and applying relevant RAI dimensions (bias... ...including data preparation, design, coding, testing, deployment, and support.Contributing to...Full timeWork experience placementLive inWork at officeLocal area
- ...Job Description Job Description Shield AI is a venture-backed defense-tech company... ...seeking a Lead Mechanical Engineer, Hardware Test to lead the design and development of... ...validation of multiple test stands used to evaluate X-BAT hardware and supporting subsystems....Full timeTemporary workPart timeWorldwide
$140k - $180k
...senior professionals ready to shape the future of AI and Data Science, your next big opportunity... ...quality, and well‑structured data for training and evaluation. Apply experimental design best practices (e.g. A/B testing, cross‑validation, randomization) to ensure statistical...Temporary work- Rex.zone is hiring a Senior Data Annotator to produce and review high-quality training data for AI systems across NLP, computer vision, and multimodal evaluation. This is a Remote, full-time role focused on annotation guidelines compliance, content safety labeling, and...Remote jobFull time
- ...automated functional validation, production health monitoring, and AI-driven failure analysis. This role is focused on ensuring... ...detection and resolution. This role sits at the intersection of test automation, production reliability, and intelligent diagnostics,...Shift work
$133.1k - $236.4k
...hands-on modeling with deep experimentation, evaluation, and analytics to improve detection and... ...into measurable business impact.Pressure-test models and data for leakage, bias, data-... ...review systems.Familiarity with LLMs and AI-assisted evaluation/analysis.Experience...Full timeTemporary workWork at officeLocal areaWorldwide3 days per week- ...high-impact team based in Seattle focused on applying advanced AI to complex, often unconventional challenges. Our work spans autonomous... ...to deployment. Key responsibilities: Design, train, and evaluate neural network architectures for real-world use Work with...Remote workFlexible hours
$130k - $260k
...data and feature pipelines, model training, evaluation, deployment, serving, monitoring, and... ...feature management, reproducibility, automated testing, observability, retraining, and model... ...of stores, the team develops advanced AI-driven solutions that empower both customers...Full timeTemporary workPart timeWork experience placement- Rex.zone is seeking a remote, full-time Senior AI Data Annotation role supporting training-data quality for modern AI/ML systems. You will deliver high-precision human feedback across RLHF, LLM evaluation, prompt evaluation, and QA evaluation to drive measurable model performance...Remote jobFull time
$198.5k - $300k
...a rare seat: you will define how modern AI—LLMs, generative AI, foundation models, and... ...systems (e.g., RAG, agentic workflows, evaluation harnesses, quality/safety controls)—ideally... ..., gender identity, results of genetic testing, veteran status, as well as any other legally...Local areaFlexible hours$143.7k - $194.4k
...build software systems powered by generative AI that serve hundreds of millions of... ...customers through design, implementation, testing, deployment, and production operations. You... ...massive datasets, scale ML models, and define evaluation criteria. Take ownership of ambiguous...Temporary workWorldwideFlexible hours- ...for the Siri App across iOS and macOS in Seattle, WA. You will lead engineers, guide architecture and performance, and drive AI-driven testing and agentic coding to elevate Siri across the operating system. In this hands-on leadership role, you will collaborate with...
$124k - $280k
...LevelSenior ManagerJob Description & SummaryThe OpportunityAs an AI & GenAI Data Scientist - Senior Manager you will leverage... ...assets that support scalable solution delivery- Evaluating model performance, testing assumptions, and adjusting approaches based on findings and...Full timeH1b
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Test & Evaluation AI SME. Be the first to apply!




