Sr. Principal Mission Engineer

Clarity Innovations is a trusted national security partner, dedicated to safeguarding our nation’s interests and delivering innovative solutions that empower the Intelligence Community (IC) and Department of Defense (DoD) to transform data into actionable intelligence, ensuring mission success in an evolving world.

Our mission-first software and data engineering platform modernizes data operations, utilizing advanced workflows, CI/CD, and secure DevSecOps practices. We focus on challenges in Information Warfare, Cyber Operations, Operational Security, and Data Structuring, enabling end-to-end solutions that drive operational impact.

We are committed to delivering cutting-edge tools and capabilities that address the most complex national security challenges, empowering our partners to stay ahead of emerging threats and ensuring the success of their critical missions. At Clarity, we are people-focused and set on being a destination employer for top talent, offering an environment where innovation thrives, careers grow, and individuals are valued. Join us as we continue to lead innovation and tackle the most pressing challenges in national security.

Position Overview

The position is for an on-site, cleared Senior Principal Mission Engineer as part of a team of software and platform engineers building a unified, multi-tenant control plane that abstracts away infrastructure differences across AWS, Azure, and on-premises environments. The platform allows application teams to provision secure, isolated Kubernetes clusters and workloads dynamically. The control plane runs Crossplane and Cluster API (CAPI).

As the primary on-site engineer, you will be dedicated to deploying, managing, and maintaining our hardware and Proxmox-based system within a secure, classified space. In this role, you will be the primary on-site technical authority responsible for the physical and virtual health of our control plane footprint. You will bridge the gap between our remote engineering pipeline and the isolated, classified hardware infrastructure, handling everything from low-level virtualization troubleshooting to direct customer engagement and user training.

The ideal candidate is highly independent, capable of driving projects from inception to completion with minimal supervision.

Key Responsibilities

• System Management & Virtualization Tuning : Maintain, update, and optimize the on-site Proxmox VE hypervisor environments, Ceph distributed storage clusters, and associated underlying server hardware.
• Troubleshooting & Incident Resolution : Diagnose and resolve complex hardware failures, kernel panics, network partitions, virtualization bottlenecks, Kubernetes pod crash loops, CNI networking failures, and node pressure conditions within the isolated classified space.
• Integration & Deployment : Deploy automated system updates, patch air-gapped systems, and integrate new physical compute or storage nodes into the local cluster environment.
• Customer Engagement & Advisory : Serve as the trusted, face‑to‑face technical representative for the program, managing day‑to‑day interactions with site stakeholders, mission owners, and local administrators.
• On‑Site Training : Conduct technical walkthroughs, operational training, and compliance briefings for customer personnel interacting with the Proxmox execution platform.
• Security & Configuration Control : Partner with the program ISSO/ISSM to enforce local security benchmarks, apply security patches, and support configuration control board reviews for physical hardware modifications.

Qualifications

• Experience : 5+ years of hands‑on experience managing enterprise Linux environments and bare‑metal hardware infrastructure in secure environments.
• Hypervisor Expertise : Strong, demonstrable experience configuring and troubleshooting Proxmox VE, KVM, QEMU, or equivalent enterprise virtualization technologies (e.g., VMware vSphere, OpenStack).
• Storage & Networking : Practical experience with Ceph storage orchestration, software‑defined networking, VLAN tagging, and low‑level Linux network troubleshooting.
• Customer‑Facing Skills : Excellent interpersonal skills with a proven track record of managing technical relationships and conducting formal training for government or military personnel.

Preferred Qualifications

• Live‑Production Incident Response : Hands‑on experience managing incident response lifecycles and resolving critical control‑plane or etcd failures within live, high‑scale production systems.
• Telemetry‑Driven Debugging : Advanced capability in utilizing telemetry, API server audit logs, distributed tracing, and metrics to perform root‑cause analysis and resolve complex issues in live production environments.

Salary Range: $100,000 - $330,000

Pay Range

The posted pay range reflects the anticipated base salary range for this position and is provided to give candidates a general understanding of the compensation opportunity. The actual salary offered will be determined based on the requirements of the position and a variety of job‑related factors, which may include geographic location, applicable Federal Government contract labor categories and wage rates, relevant experience, skills and competencies, education, certifications, and the scope and responsibilities of the role.

Because this position may be performed in multiple locations and may be subject to different contract requirements, the salary ultimately offered may vary based on the applicable location and contract. The posted range is not a guarantee of a specific starting salary or an indication that an offer will be made at any particular point within the range.

Total Rewards

In addition to competitive base pay, Clarity provides a Total Rewards package designed to attract and retain exceptional professionals committed to excellence. Our benefits reflect the value we place on skilled individuals who drive our impact in critical digital missions.

At Clarity, a career means joining a high-performing team advancing vital work in the digital landscape. We encourage you to consider the total value of joining us—competitive compensation, comprehensive benefits, and resources that support your growth and impact.

Our Total Rewards package includes performance‑based bonuses, extensive medical, dental, and vision coverage, a 401(k) plan with company match, substantial paid time off and holidays, flexible work arrangements where possible, and a range of income protection benefits. We are committed to supporting your well‑being, professional growth, and ability to contribute to essential digital missions.

We are an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or veteran status.