Azure Databricks Infrastructure Engineer

Azure Databricks Infrastructure EngineerLocation - Remote Position SummaryWe are seeking an experienced Azure Databricks Infrastructure Engineer to execute migrations of Azure Databricks environments between Azure tenants. The successful candidate will have deep expertise in Databricks, Azure cloud services, identity and access management, networking, data engineering, and platform modernization.This role is responsible for executing a migration strategy that minimizes risk, downtime, and business disruption. The ideal candidate will combine strong technical capabilities with consulting skills and be comfortable interacting with architects, platform teams, security organizations, and business stakeholders.Key ResponsibilitiesMigration PlanningAnalyze tenant-specific dependencies, networking requirements, DNS, Private Endpoints, and identity integrations.Define migration waves and cutover strategies.Databricks Migration ExecutionMigrate Databricks workspaces between Azure tenants.Rebuild or migrate:· Workspaces· Compute policies· Job schedules· Repositories· Workflows· Secret scopes· Access controlsMigrate notebooks, libraries, packages, and configuration artifacts.Validate Unity Catalog and data governance configurations.Implement Infrastructure as Code (IaC) and automation where applicable.Security and IdentityMigrate and validate:Microsoft Entra ID integrationsManaged identitiesService principalsRBAC assignmentsDatabricks groups and permissionsCoordinate Key Vault migration and secret rotation activities.Ensure compliance with the Principle of Least Privilege (PLP).Support security reviews and governance approvals.Data MigrationAssess and migrate associated datasets and storage integrations.Validate Delta Lake tables, metadata, and schemas.Develop rollback and recovery procedures.Automation and DevOpsDevelop Infrastructure-as-Code solutions using:TerraformBicepARM TemplatesAutomate migration and validation activities wherever possible.Required Technical SkillsAzureAzure DatabricksMicrosoft Entra IDAzure Virtual Networks (VNets)Private EndpointsAzure Key VaultAzure Storage AccountsAzure Data Lake Storage Gen2 (ADLS Gen2)Azure MonitorLog AnalyticsAzure PolicyAzure RBACDatabricksWorkspace AdministrationUnity CatalogDelta LakeDatabricks ReposMLflowCluster ManagementCluster PoliciesSecret ScopesDatabricks REST APIsAutomation & DevOpsTerraformAzure DevOpsGitHub ActionsPowerShellAzure CLIDatabricks CLIRequired Experience7+ years of Azure cloud experience.5+ years of Azure Databricks administration and engineering experience.Demonstrated experience migrating Azure resources across tenants.Experience with large-scale data platform migrations.Experience designing secure cloud landing zones.Experience implementing Infrastructure as Code (IaC).Experience working in regulated enterprise environments.Preferred QualificationsAzure Solutions Architect Expert certification.Databricks Administrator Certification.Databricks Solution Architect Certification.Experience with Unity Catalog migrations.Experience with mergers, acquisitions, divestitures, or tenant-to-tenant transformations.DeliverablesThe resource should be capable of independently producing:Migration strategy and wave plan.Detailed migration runbooks.Cutover plans.Rollback plans.Operational handover documentation.Ideal Candidate ProfileA senior-level architect or lead engineer who has previously executed Azure Databricks tenant-to-tenant migrations, understands both platform and data engineering concerns, can engage with executive stakeholders, and can lead migration activities from assessment through production cutover with minimal supervision.