Lead Data Platform Engineer - AWS Databricks Lakehouse

Job Family Data Engineering & Architecture Consulting Travel Required Up to 10% Clearance Required Active Secret What You Will Do As a Manager in Guidehouse’s AI & Data practice, you will serve as a hands‑on technical leader and delivery manager designing and implementing modern cloud‑native data platforms for federal clients, with a primary emphasis on AWS and Databricks. You will lead architecture decisions, mentor engineers, and ensure high‑quality, secure, and scalable delivery aligned to mission outcomes. Key responsibilities include: Architect and lead delivery of AWS-based lakehouse solutions using Databricks, including integration with AWS-native services such as S3, Glue, Lambda, IAM, and Redshift where appropriate. Design, build, and optimize ETL/ELT pipelines leveraging Databricks, Apache Spark, Delta Lake, Python, and SQL, supporting batch and streaming use cases. Define and enforce data architecture standards, including ingestion patterns, layered/medallion designs, schema evolution, performance tuning, and cost optimization. Implement production‑grade engineering practices, including data quality validation, reconciliation, logging, monitoring, and operational alerting. Apply data security and governance controls appropriate for federal and defense environments, including role‑based access, controlled data handling, and auditability. Partner with client stakeholders and product owners to translate mission and business requirements into technical designs, delivery roadmaps, and implementation plans. Lead and mentor data engineers through code reviews, architecture reviews, and delivery execution; contribute to reusable accelerators and reference architectures. Establish and mature CI/CD pipelines for Databricks workloads and supporting infrastructure, improving reliability, repeatability, and compliance of releases. What You Will Need US Citizenship required. An ACTIVE and MAINTAINED "SECRET" Federal or DoD security clearance. Bachelor’s degree obtained or equivalent experience. Seven (7+) years of progressive experience in data engineering, data platform delivery, or data architecture roles, including leading technical teams or workstreams. Strong hands‑on experience with AWS data platforms, including designing and operating production systems leveraging Databricks on AWS. Deep expertise with Apache Spark, Databricks, Delta Lake, Python, and SQL, including performance tuning of distributed workloads. Experience designing scalable, secure data architectures that support analytics, reporting, and advanced use cases (e.g., AI/ML enablement). Familiarity with Agile delivery methods and proven ability to collaborate effectively with cross‑functional and client teams. Strong written and verbal communication skills, including the ability to explain complex technical topics to non‑technical stakeholders. What Would Be Nice To Have Experience delivering Azure-based data platforms, including Azure Databricks, ADLS Gen2, Azure Data Factory, or Synapse Analytics. Bachelor’s degree obtained. Computer Science, Engineering, Information Systems, or a related technical field. Experience with Databricks Unity Catalog, including permissions management and governance enablement. Infrastructure‑as‑Code experience (e.g., Terraform) and advanced CI/CD automation for data platforms. Experience with streaming and event‑driven architectures (e.g., Kafka, Kinesis). Cloud or platform certifications (AWS, Databricks, or Azure). Prior experience supporting DoD or Defense & Security federal clients. Salary The annual salary range for this position is $106,000.00–$176,000.00. Compensation decisions depend on a wide range of factors, including but not limited to skill sets, experience and training, security clearances, licensure and certifications, and other business and organizational needs. What We Offer Guidehouse offers a comprehensive, total rewards package that includes competitive compensation and a flexible benefits package that reflects our commitment to creating a diverse and supportive workplace. Benefits include: Medical, Rx, Dental & Vision Insurance Personal and Family Sick Time & Company Paid Holidays Position may be eligible for a discretionary variable incentive bonus Parental Leave and Adoption Assistance 401(k) Retirement Plan Basic Life & Supplemental Life Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts Short‑Term & Long‑Term Disability Student Loan PayDown Tuition Reimbursement, Personal Development & Learning Opportunities Skills Development & Certifications Employee Referral Program Corporate Sponsored Events & Community Outreach Emergency Back‑Up Childcare Program Mobility Stipend Equal Opportunity Employer Guidehouse is an Equal Opportunity Employer–Protected Veterans, Individuals with Disabilities or any other basis protected by law, ordinance, or regulation. Criminal History Guidehouse will consider for employment qualified applicants with criminal histories in a manner consistent with the requirements of applicable law or ordinance including the Fair Chance Ordinance of Los Angeles and San Francisco. #J-18808-Ljbffr