Databricks SME

Duration: 18 months Location: Washington, D.C. 20224 Key Responsibilities Serve as the Databricks SME and guide technical decisions for workspace architecture design, configuration, and optimization. Lead governance processes within Databricks environments, including access controls, data lineage, and workspace policies. Optimize Databricks cluster and job performance; identify and resolve bottlenecks in data pipelines. Design and implement secure integration patterns between Databricks and AWS services (e.g., S3, IAM, Glue, Redshift). Perform specialized operations including Delta Lake management, Unity Catalog administration, and MLflow tracking. Collaborate with government technical leads on architecture reviews and platform enhancement initiatives. Develop and maintain technical documentation, runbooks, and best practice guidelines. Support platform security requirements and ensure compliance with IRS/Treasury policies. Mentor mid-level engineers and contribute to team knowledge sharing. Participate in code reviews, design discussions, and agile ceremonies. Required Qualifications & Skills 8+ years of hands‑on experience in cloud technology environments. 3–5 years of dedicated Databricks experience including workspace administration and optimization. Proven expertise as a Databricks SME with Unity Catalog, Delta Lake, and cluster management. Strong proficiency with AWS services including S3, IAM, Glue, Redshift, and VPC networking. Experience with data governance frameworks and security controls in cloud environments. Proficiency in Python, Scala, or SQL for data engineering tasks. Familiarity with federal IT compliance standards (FedRAMP, FISMA) preferred. Knowledge of tax‑processing or financial data management systems is highly preferred. Databricks Certified Data Engineer Professional certification preferred. AWS Certified Data Analytics or Solutions Architect certification preferred. #J-18808-Ljbffr Agelix Consulting LLC