Senior Platform Engineer - Databricks Integration

Senior Platform Engineer – Databricks Integration Everforth ECS is seeking a Senior Platform Engineer – Databricks Integration to work in the National Capital Region covering the Pentagon, Falls Church, and Fairfax . This position is contingent upon contract award. The War Data Platform (WDP) is a key initiative within the U.S. Department of War's (DoW) AI-First strategy introduced in early 2026. The WDP focuses on operational warfighting data and aims to accelerate the deployment of artificial intelligence (AI) on the battlefield. The WDP extends to Unclassified, Secret, and Top Secret environments, and supports collaboration between Combatant Commands, Joint Staff directorates, Senior Executive Service leaders, and operational analysts. Responsibilities The Senior Platform Engineer - Databricks Integration provides senior-level technical leadership for the integration and operational advancement of Databricks capabilities across WDP's full multi-enclave architecture, spanning Unclassified/NIPRNet, Secret/SIPRNet, and Top Secret/JWICS environments. This role is central to enabling high-performance distributed data processing, MLOps maturity, and mission-aligned data exploitation in direct support of WDP Core Integration objectives. Leads the integration, configuration, and operational advancement of Databricks platform capabilities supporting War Data Platform (WDP) Core Integration mission requirements across Unclassified and NIPR, Secret and SIPR, and Top Secret and JWICS enclaves. Designs distributed data pipeline architectures that ingest, transform, and process high-volume structured and unstructured data using Databricks Workflows, Delta Lake storage patterns, Unity Catalog governance rules, and medallion-layered data modeling aligned with enterprise architecture objectives. Implements MLOps frameworks using Databricks Model Registry, Feature Store, and automated training pipelines to strengthen machine learning reproducibility, deployment readiness, and mission analytics responsiveness. Optimizes Databricks cluster performance, auto-scaling behaviors, job concurrency, and cloud cost allocations using monitoring utilities, workload metrics dashboards, and predictive capacity planning. Integrates the Databricks environment with existing WDP Core Integration platform components including API gateways, cross-domain data movement tools, orchestration systems, data governance platforms, and enclave security controls. Conducts technical assessments of emerging commercial and open-source technologies and leads structured tool evaluations to determine suitability for integration into the WDP Core Integration ecosystem. Provides senior-level engineering mentorship to data engineers, analytic developers, and artificial intelligence teams, raising maturity in distributed computing practices, performance tuning, and data security. Produces architectural diagrams, integration guides, performance reports, and compliance documentation that advance platform interoperability, scalability, and mission-aligned data exploitation capabilities. Performs other duties as assigned. Required Skills Current Secret security clearance with the ability to obtain and maintain a Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI). 10 or more years of progressive experience in platform or data engineering, with demonstrated hands‑on expertise designing and operating Databricks environments at enterprise scale, including Delta Lake, Unity Catalog, Databricks Workflows, Model Registry, and Feature Store. Proven experience architecting and deploying distributed data pipelines and MLOps workflows across multi-enclave or multi-classification environments, with working knowledge of cross-domain data movement constraints and enclave security controls. Demonstrated experience integrating Databricks with complementary platform components such as API gateways, workflow orchestration tools, data governance platforms, and cloud-native storage solutions in AWS GovCloud or equivalent classified cloud environments. Experience operating within DoW-compliant DevSecOps frameworks, including the use of CI/CD pipelines, Infrastructure-as-Code (IaC), and agile development methodologies to deliver platform capabilities in iterative, mission-driven release cycles. Strong problem‑solving and decision‑making capabilities, with a proven ability to weigh the relative costs and benefits of potential actions and identify the most appropriate solution. Highly developed interpersonal and oral/written communication skills, with the ability to effectively and professionally interact with a diverse set of stakeholders (from peers to end‑users to executive management). Desired Skills Active Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI) eligibility. Databricks Certified Data Engineer Professional, Databricks Certified Machine Learning Professional, or an equivalent advanced certification demonstrating validated expertise in large-scale Databricks platform engineering and MLOps. Familiarity with complementary data processing and orchestration tools used within the WDP ecosystem, such as Apache Airflow, Apache Kafka, Apache NiFi, Ray, or vLLM, and the ability to integrate these tools with Databricks in a multi-enclave environment. Working knowledge of Zero Trust Architecture principles and Risk Management Framework (RMF) requirements as they apply to data platform governance, Unity Catalog access controls, and continuous monitoring in DoW-compliant cloud environments. Experience supporting AI/ML pipeline development using open-source frameworks such as PyTorch, TensorFlow, or MLflow in a DoW or Intelligence Community cloud context, with demonstrated ability to bridge platform engineering and data science team workflows. ECS Federal LLC is an equal opportunity employer and does not discriminate or allow discrimination on the basis of any characteristic protected by law. All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, or local jurisdiction law. #J-18808-Ljbffr ECS