AWS Databricks Platform Administrator

Job Summary

We are seeking a Senior Databricks Platform Engineer with strong hands-on experience in administering, securing, optimizing, and supporting Databricks environments on AWS. The role focuses on platform reliability, security, governance, automation, cost optimization, and production support for enterprise data platforms.

Key Responsibilities

• Administer Databricks workspaces, Unity Catalog, clusters, SQL warehouses, jobs, secrets, and access controls.
• Manage AWS integrations, including S3, IAM, VPC, KMS, PrivateLink, CloudWatch, and CloudTrail.
• Implement security, identity management, data governance, and access control policies.
• Automate platform provisioning and operations using Terraform, Python, Databricks CLI/SDK, and CI/CD pipelines.
• Monitor platform performance, troubleshoot production issues, manage incidents, and ensure SLA compliance.
• Optimize Databricks and AWS resource utilization and costs.
• Manage platform upgrades, disaster recovery, compliance, and operational documentation.
• Collaborate with cloud, security, and data engineering teams and provide technical guidance.

Required Qualifications

• 9+ years of overall IT experience, with at least 5 years of hands-on Databricks administration experience.
• Strong knowledge of Databricks on AWS, Unity Catalog, Delta Lake, Apache Spark, clusters, jobs, and SQL warehouses.
• Hands-on experience with AWS services, including S3, IAM, VPC, KMS, PrivateLink, CloudWatch, and CloudTrail.
• Experience with Terraform, Python, shell scripting, REST APIs, Databricks CLI/SDK, and CI/CD.
• Strong understanding of cloud security, networking, monitoring, performance tuning, FinOps, and production support.
• Bachelor's degree in Computer Science, IT, Engineering, or equivalent relevant experience.