Staff Engineer, Test Automation

Responsibilities

• Own the technical strategy, architecture, and roadmap for automated testing, verification, and MLOps quality across the Hivemind ecosystem.
• Design and maintain scalable test frameworks for autonomy software, backend services, APIs, operator-facing applications, and distributed hardware environments.
• Lead functional, integration, regression, system, performance, reliability, and end-to-end testing across simulation, edge-compute, software-in-the-loop, and hardware-in-the-loop environments.
• Build automated ML validation pipelines covering data quality, training reproducibility, model accuracy, robustness, regression, latency, resource utilization, and system integration.
• Establish versioning and traceability for datasets, models, configurations, evaluation results, and deployment artifacts through continuous training workflows.
• Develop scenario-based validation for autonomy models, including degraded sensing or communications, distribution shifts, edge cases, and representative mission conditions.
• Create observability, analytics, and failure-triage capabilities for software behavior, model and data drift, inference health, test results, and production performance.
• Build Python automation, test harnesses, simulators, stubs, mocks, and synthetic-data capabilities to improve test execution, coverage, reporting, and developer productivity.
• Collaborate with software, autonomy, machine learning, data, simulation, and systems engineers to define verification strategies and improve designs.
• Develop and govern AI-assisted engineering workflows using coding agents and LLM-based tools for test generation, log analysis, debugging, and failure triage.

Requirements

• Typically 8+ years of relevant experience in software engineering, test infrastructure, developer tooling, MLOps, systems integration, or systems verification, or an equivalent combination of experience and demonstrated impact.
• At least 5 years of experience building scalable automation frameworks or developer tooling in Python.
• Demonstrated success designing test, CI/CD, or MLOps infrastructure used across multiple engineering teams.
• Experience validating machine learning systems across data, training, evaluation, packaging, deployment, and monitoring lifecycles.
• Experience with ML quality risks, model-performance baselines, automated evaluation suites, release thresholds, and candidate-to-production comparison workflows.
• Experience testing GPU-accelerated workloads and multi-tenant Kubernetes environments, including scheduling, allocation, utilization, resource contention, RBAC, quotas, isolation, and scheduling behavior.
• Experience with performance benchmarking, profiling, observability, integrated hardware and software qualification, distributed systems, backend services, APIs, and regression strategies.
• Strong asynchronous and concurrent Python programming skills for scalable automation and parallel test execution.
• Experience with package and dependency management, reproducible environments, build systems, automated observability, log collection, analytics, reporting, root-cause analysis, and Linux-based development environments.
• Preferred qualifications include experience with MLflow, Kubeflow, Weights & Biases, SageMaker, Vertex AI, GPU orchestration and profiling tools, NVIDIA GPU infrastructure, autonomy or robotics models, embedded or edge-compute platforms, production hardware qualification, fault-injection and recovery testing, containers, Kubernetes, cloud infrastructure, infrastructure as code, Go or TypeScript, C or C++ interoperability, and safety-critical verification standards such as DO-178C and MIL-STD-882.

Benefits

• Full-time employees receive pay within the listed range plus bonus, benefits, and equity.
• Temporary employees receive pay within the listed range plus a temporary benefits package applicable after 60 days of employment.
• Offers are contingent on a cleared background and possible reference check.
• Military fellows and part-time employees are not eligible for benefits.
• Shield AI provides equal employment opportunity and disability or special-needs accommodation.