Senior Data Engineer

EMPLOYER IS A CONTRACTOR FOR THE U.S. GOVERNMENT. THIS POSITION WILL REQUIRE U.S. CITIZENSHIP. Role Description This is a forward-deployed engineering role. You'll work shoulder-to-shoulder with mission heroes: the users, operators, and program engineers who run real workloads in real DoD environments. You'll be the person who takes UDS Data Capability from "deployable" to "running in production under classification." You will deploy, harden, integrate, and operate our data stack in the mission hero's environment, then carry what you learn back into the product so the next engagement is easier. The scope of this work varies by engagement. Some programs require standing up a governed data platform to centralize sensor or operational data from known sources, where the problem is well-defined and the priority is building and shipping. Others involve consolidating data across hundreds of interconnected government systems of record with overlapping schemas, deeply interdependent data flows, and significant architectural complexity. You should be equally comfortable executing against a defined data problem and navigating ambiguity in a complex data landscape where the right approach isn't yet clear. Forward-deployed means aligned with the mission hero, not always on a plane. Most of the work is remote. You'll travel to sites when the engagement calls for it: initial standup, integration work, incident response, training and knowledge transfer. The rest of the time, you'll be on a video call or in a shared chat with that same mission hero. The role is defined by the relationship with the mission hero, not by where you sit. You are a data engineer who can own the full breadth of a data platform (storage, ingestion, streaming, governance, and access) and deliver working solutions in constrained, mission-critical environments. About 20% of your time, you'll also invest in the forward-deployed team itself, pairing with junior Data Engineer FDEs on hard problems and helping them develop the technical judgment that only comes from field experience. Responsibilities Deploy and harden UDS Data Capability in the mission hero's environment. Stand up the UDS Store (Iceberg, Rook/Ceph, pgvector, Postgres), wire up UDS Transit for air-gap data movement, configure UDS Govern policies (Pepr/Lula), and integrate UDS Connect (Strimzi/Kafka) where streaming or legacy connectors are required. Own the integration with existing mission systems. Connect UDS Data Capability to legacy databases, flat-file drops, SOAP/REST endpoints, message buses, existing object storage, and identity providers (Keycloak, mission-side SSO). Prior experience integrating with these types of systems is more relevant than experience building on them. Map and navigate complex data landscapes. Some engagements involve hundreds of interconnected systems of record with overlapping schemas and deeply interdependent data flows. You'll need to trace how data moves across systems, identify dependencies, and advise government stakeholders on consolidation and architecture decisions. Build pipelines that move data through classification boundaries, including ingestion, transformation, catalog registration, model/dataset packaging via Zarf, cross-domain transit, and eventual consistency across DDIL conditions. Establish data provenance, lineage, and governance practices. Track where data came from, how it transformed, and who can access it. Operate what you deploy. Initial day-2 ownership includes capacity, performance, backup/restore (Velero), observability (Vector/Loki), incident response, and upgrade paths. Hand off to the mission hero's ops team once it's stable. Generate accreditation artifacts, including STIG evidence, cATO documentation, FIPS validation notes, and policy mappings. You produce the evidence the mission hero's ISSM/ISSO needs to run this in IL4/IL5. Be the voice of the mission hero back to product and engineering. File issues, write postmortems, propose operator improvements, and ensure field experience directly informs platform development. Train and transfer. Leave the mission hero's team self-sufficient through runbooks, architecture docs, working sessions, and knowledge transfer. Grow junior Data Engineer FDEs. Pair on hard problems, review integration designs before they reach the customer, and accelerate technical development across the team. This is a mentorship role, not a management one. Preferred Locations: Colorado Springs, CO or Florida Space Coast (Cape Canaveral / Cocoa Beach / Melbourne). Remote candidates will be considered, but local presence is strongly preferred. Travel Expectations: Candidates based in Colorado Springs or the Florida Space Coast can expect on-site presence 1-2 days per week. Remote candidates should expect to travel to the engagement site for approximately one week every 4-6 weeks. The listed responsibilities are not exhaustive and additional responsibilities may be assigned based on the evolving needs of the organization. We are seeking a dynamic individual who is able to adapt and take on new responsibilities as they arise. Required Qualifications Data engineering breadth Unstructured data at scale. Production experience storing and querying large unstructured datasets using data lake architectures. Spark strongly preferred. Streaming & integration. Building and operating stream processing infrastructure (Kafka, Redpanda, Flink, or equivalent) and bridging data from heterogeneous sources into modern pipelines. Data warehousing. Open-source data warehousing platform experience. These environments do not support proprietary platforms, so you need to be comfortable building without them. Pipelines & orchestration. Airflow, Dagster, Argo Workflows, or similar. Comfort building, scheduling, monitoring, and recovering production data pipelines. Data modeling & SQL. Fluent in SQL. Comfortable designing schemas for both analytical and operational workloads. Open-source orientation. You are comfortable building on open-source tooling and contributing back to it. What "forward-deployed" requires U.S. citizenship and the ability to obtain and maintain a DoD security clearance. Clearance sponsorship available for the right candidate. Comfort being the technical face of Defense Unicorns to a mission hero. Clear communication with both technical and non-technical stakeholders. Comfort with periodic on-site work, sometimes for days at a stretch, and equal comfort working remotely. Bias toward delivery. Preference for shipping a working integration over perfecting a design that hasn't met a real workload. Self-direction. You will encounter environments and problems that are not yet documented and will need to work through them independently. Willingness to mentor junior engineers and help them build technical judgment through hands-on experience. Preferred Qualifications Data provenance, lineage, and governance. Experience with lineage tracking, data catalogs, provenance systems, or governance frameworks. Depth here will be weighted heavily. DoD or defense program experience. Active Secret clearance (or higher). Lakehouse & storage: Apache Iceberg (or Delta/Hudi), object storage (Ceph/S3-compatible), Postgres (including extensions like pgvector), columnar/OLAP engines (Trino, DuckDB, ClickHouse, Spark SQL). Change Data Capture: Debezium or similar CDC patterns. Governance, catalog & access: REST catalogs (Iceberg REST, Polaris/Gravitino/Nessie family), ABAC/RBAC patterns, OIDC/OAuth, lineage and audit. Kubernetes awareness. Deep K8s expertise is not required; general familiarity with deployments, operators, and how applications run on Kubernetes is valuable. Linux fundamentals, container runtime behavior, networking, TLS, secrets management. IaC (Terraform, Pulumi, or similar) and GitOps patterns (Flux, ArgoCD). Familiarity with the CNCF ecosystem, including the distinction between foundation projects and single-vendor projects. AI/ML awareness. General understanding of how data infrastructure supports model training, versioning, provenance, and AI operations. Familiarity with Air Force or Space Force systems of record (e.g., MILPDS, ARMS) and how data flows between them. You do not need to meet every preferred qualification. You do need to be credible across the core data engineering requirements. Full compensation packages are based on candidate experience. Compensation ranges are established using national benchmarking data and apply across all geographic locations within the United States. Remote - USA $148,750 - $201,250 USD Who We Are Defense Unicorns delivers mission value by streamlining software delivery so our customers can focus on the most important challenges. We share a vision of freedom and security for the advancement of progress and innovation. Our commitment to this vision, and to our mission-driven customers, means a commitment to speed, user experience and optionality, without compromising security. Our team is composed of innovators, software engineers, and veterans with decades of experience delivering technology programs across the federal market. What We Do We create and deliver secure solutions for continuous software integration and delivery. Defense Unicorns consolidates the best practices for security pipelines, testing, and deployment automation in order to meet the high security requirements valued by mission owners. Our solutions are agnostic by design and we believe that growing a robust ecosystem of secure, cloud-native software solutions can help enterprise customers inside and outside the federal market buy and integrate software more easily. Who We Serve Defense Unicorns’ customers are mission-focused leaders across public and private enterprises. We proudly support defense and civil agencies across the U.S. government and we work closely with the creators of leading-edge software solutions to deliver value to the mission-owner by improving the security and consumability of commercial software products. What We Work On Kubernetes Cloud Environments (AWS/GCP and Azure) Infrastructure-as-code (like Terraform/Pulumi) Continuous Delivery and automation tooling GitOps Containers CNCF projects and open source products and packages Helm/Kustomize-Value Stream Mapping Building and improving security delivery Building Kubernetes and cloud native applications Benefits Our Unicorns Enjoy Health: Medical/Dental/Vision Premiums are 100% Company Paid Health Savings Account Life Insurance Disability Insurance Financial: 401k Retirement Plan Company Stock Options Home Office Budget Leave: We offer all full-time Unicorns Flexible Time Off (FTO) plus all Federal Holidays, one week for Thanksgiving, and two weeks for Christmas and New Year’s Paid Parental Leave Learning: Reimbursement for approved trainings/subscriptions Conferences (travel, lodging, and fees) Don’t have all the preferred experience or qualifications? Studies show that underrepresented groups like women and people of color are less likely to apply to jobs if they don't meet every requirement listed. At Defense Unicorns, we're committed to diversity. If you're enthusiastic about the role but don't match every criteria, we encourage you to apply. You could be the perfect fit for this or another role! Defense Unicorns is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, disability, sex, sexual orientation, gender identity or expression, age, national origin, veteran status, genetic information, union status and/or beliefs, or any other characteristic protected by federal, state, or local law. CCPA DISCLOSURE