Data Lake Engineer Jobs

Data Lake Engineer Prescient Edge is seeking a Data Lake Engineer to support a Federal Government client. Benefits include a competitive salary with performance bonus opportunities, comprehensive healthcare benefits, a substantial retirement plan, career development opportunities, and a positive work environment. Description: Design, implement, and maintain scalable Data Lake architectures to support structured and unstructured data ingestion, ensuring efficient data access and retrieval. Configure and manage the integration interface between the Data Lake and the managed knowledge graph platform (Stardog), including SPARQL endpoint access, metadata federation, and catalog alignment. Follow access control policies and usage scope. Confirm compliance with access policies on a quarterly basis and document the results in the Data Governance & Compliance Report. Optimize ETL pipelines for high-volume data transformation, ensuring compliance with DoD IL-4/IL-5 security standards. Implement storage tiering strategies and access controls, ensuring data is properly classified, retained, and accessed per DoD governance requirements. Submit the Data Lake Performance & Optimization Report, detailing ingestion efficiency, access control improvements, and storage utilization metrics. Job Requirements Experience: Possess the knowledge and capability to support Data Lake platform administration and enterprise data architecture for DoD data-driven projects. Skilled in Data Lake Platform administration, including workspace management, cluster optimization, and Unity Catalog integration for secure data governance. Proficient in ETL pipeline development, Delta Lake architecture, and Data Lakehouse optimization for DoD analytics and mission critical data workflows. Experienced in SysEngOps, DevSecOps, version control systems (Git), and CI/CD pipelines to streamline Data Lake development and deployment. Knowledgeable in identity and access management (IAM), role-based access control (RBAC), and cloud security best practices across AWS, Azure, and GCP. Hands-on expertise in Python, SQL/NoSQL, Apache Spark, Databricks SQL, Terraform, and cloud native data services for large-scale data processing and analytics. Has strong background in: Databricks platform administration, including workspace configuration, Unity Catalog, cluster performance tuning, and cloud integration. Data pipeline development (ETL/ELT), Delta Lake optimization, and AI/ML workflow integration. SysEngOps, DevSecOps, version control systems (Git), and CI/CD pipelines for automated deployment and governance. Cloud security, role-based access control (RBAC), and compliance with DoD data policies across AWS, Azure, and GCP. Education: A Master's degree or higher (e.g. Ph.D.) in Computer Science, Information Technology, Systems Engineering, Data Science, Business Administration, Engineering Management, or a closely related field; or a minimum of 11+ years of experience managing complex technical projects in enterprise data architecture, Databricks administration, and cloud-based data platforms. Security clearance: Active TS/SCI clearance. Location: Doral, Florida. (USSOUTHCOM Headquarters) Prescient Edge is a Veteran-Owned Small Business (VOSB) founded as a counterintelligence (CI) and Human Intelligence (HUMINT) company in 2008. We are a global operations and solutions integrator delivering full-spectrum intelligence analysis support, training, security, and RD&E support solutions to the Department of Defense and throughout the intelligence community. Prescient Edge is an Equal Opportunity Employer (EEO). All applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other characteristic that is protected by law. We strive to foster equity and inclusion throughout our organization because we believe that diversity of thought is critical for creating a safe and engaging work environment while also enabling the organization's success. ClearanceJobs