Databricks Database Administrator

Role Description

We are looking for a talented, experienced individual to join our Data Operations team and lead the company’s migration from SQL Server to DataBricks. The DBA is a “hands on” position requiring the candidate to be able to work with applicable technologies to develop and maintain databases across the organization, while ensuring high levels of data availability. The ideal candidate will possess strong experience in Databricks Lakehouse architecture, Azure cloud services, automation, security, and platform operations within a healthcare or regulated environment.

Essential Duties & Responsibilities

• Modernize the data warehouse environment by migrating the platform to Azure Databricks.
• Administer and maintain Databricks workspaces across development, test, and production environments.
• Configure and manage clusters, cluster policies, instance pools, SQL Warehouses, and job scheduling.
• Monitor platform utilization, performance, and capacity planning.
• Optimize cluster configurations for cost, performance, and reliability.
• Manage workspace configurations, notebooks, libraries, and package dependencies.
• Experience implementing Lakehouse architecture and Medallion data patterns (Bronze, Silver, Gold).
• Implement and maintain RBAC, Unity Catalog, workspace permissions, and data access controls.
• Collaborate with Azure platform teams to manage Azure Key Vault, Azure Virtual Networks, Private Endpoints, Managed Identities, Azure Monitor, and Log Analytics.
• Troubleshoot cloud connectivity, networking, and authentication issues.
• Ensure secure connectivity between Databricks and enterprise systems.
• Support Data Engineering, Analytics, AI/ML, and Data Science teams.
• Establish operational processes for environment provisioning, support, monitoring, and incident management.
• Perform and/or coordinate installations, upgrades, and patches to Microsoft SQL products and tools.
• Develop Infrastructure as Code (IaC) solutions using Terraform and CI/CD pipelines.
• Automate workspace provisioning, user onboarding, cluster deployments, and policy management.
• Integrate Databricks deployments with Azure DevOps or GitHub Actions.
• Manage source control integration and release processes.
• Provide On-call Support during off-hours for Production, test, and development DBs based on criticality.

Qualifications

• 5+ years of experience administering enterprise data platforms.
• 3+ years of hands-on Databricks administration experience.
• Strong experience with Azure Databricks, Unity Catalog, Delta Lake, Azure Data Lake Storage (ADLS), Azure Active Directory (Entra ID), Azure Networking.
• Experience implementing RBAC, security policies, and governance controls.
• Strong knowledge of monitoring, logging, troubleshooting, and performance tuning.
• Experience managing production environments and platform support operations.
• Proficiency in Python, SQL, and scripting languages.
• Experience with Infrastructure as Code (Terraform preferred).
• Has familiarity with Microsoft Azure and provisioning of SQL Server Databases.
• Understanding of HIPAA, SOC 2, or other compliance frameworks relevant to healthcare data.
• Bachelor’s degree from an accredited college or university or equivalent experience.