Azure Databricks Administrator

 IT/Health IT/InformaticsNew York • ID:

1166782_RR00124352

• Full-Time/RegularNYU Langone Health is a fully integrated health system that consistently achieves the best patient outcomes through a rigorous focus on quality that has resulted in some of the lowest mortality rates in the nation. Vizient Inc. has ranked NYU Langone the No. 1 comprehensive academic medical center in the country for three years in a row, and U.S. News & World Report recently placed nine of its clinical specialties among the top five in the nation. NYU Langone offers a comprehensive range of medical services with one high standard of care across 6 inpatient locations, its Perlmutter Cancer Center, and over 320 outpatient locations in the New York area and Florida. With $14.2 billion in revenue this year, the system also includes two tuition-free medical schools, in Manhattan and on Long Island, and a vast research enterprise with over $1 billion in active awards from the National Institutes of Health. For more information, go toNYU Langone Health, and interact with us on LinkedIn, Glassdoor, Indeed, Facebook, X, YouTubeandInstagram.Position Summary:We have an exciting opportunity to join our team as a Azure Databricks Administrator. As an Azure Databricks Administrator within this department, you will be responsible for managing and maintaining the Databricks platform, ensuring optimal performance and security across workspaces, clusters, and jobs. This role is crucial in overseeing user administration, managing Azure storage solutions, and implementing CI/CD pipelines using Azure DevOps. By leveraging a deep understanding of Databricks architecture and proficiency in scripting languages, the administrator will automate tasks and enhance the efficiency of data operations. Strong communication skills and a commitment to high-quality support will enable effective collaboration with cross-functional teams, directly contributing to the department's mission of delivering robust data solutions and insights.Job Responsibilities:Manage and maintain Azure Databricks Platform, workspaces, clusters, and jobsOversee user administration including access controls and permissionsHandle library installations, runtime management, and policy enforcementImplement/analyze cost control measures and monitor platform usage and performanceAdminister Unity Catalog for data governance and securityCollaborate with data engineers, data scientists, and analysts to optimize and streamline data workflows, and analytical pipelines on the Databricks platformManage Azure storage solutions, including Blob Storage and Data Lake StorageAdminister Azure Key Vault for secure storage of secrets and keysConfigure and manage Azure Data Factory for data integration and ETL processesImplement and manage VNETs, firewalls, Azure policies and security best practicesImplement and manage Azure DevOps for CI/CD pipelines and version controlImplement best practices for automated deployment and continuous integrationSet up budgets and alerts to monitor and control Azure costs and spendImplement monitoring solutions for platform health and performanceConfigure alerts for proactive issue detection and resolutionDatabricks Lakehouse Monitoring develop and execute ad hoc operational reports not already implemented within the EDC (e.g., nonstandard query reports).Minimum Qualifications:Bachelor's degree in Computer science, Information systems, or Engineering7+ years of related professional experience administering data platforms3+ years experience with Azure Databricks Lakehouse PlatformProficiency in scripting languages such as Python/PySpark, PowerShell, or BashExperience in automating administrative tasks and workflowsKnowledge of security best practices and compliance requirementsExperience with ETL processes, data pipelines, and big data technologiesExperience with backup and restore procedures for Databricks and Azure servicesAbility to troubleshoot and resolve issues in a timely and efficient mannerStrong verbal and written communication skillsAbility to document processes, procedures, and configurations clearlyCommitment to providing high-quality support to internal/external users and stakeholdersAbility to understand and address the needs of internal and external customersTeam player with the ability to work collaboratively with cross-functional teamsFlexibility to adapt to changing requirements and prioritiesWillingness to learn new technologies and continuously improve skillsPreferred Qualifications:Deep understanding of Databricks architecture, features, and best practicesExperience with Delta Lake and Databricks SQL for advanced data managementExperience in performance tuning and optimization for both Databricks and Azure environmentsExperience supporting/administering data science and ML workloads in DatabricksKnowledge of disaster recovery planning and implementationExperience integrating Databricks with other data platforms and toolsUnderstanding of hybrid and multi-cloud environmentsExperience implementing data protection measures and ensuring regulatory complianceQualified candidates must be able to effectively communicate with all levels of the organization.NYU Langone Health provides its staff with far more than just a place to work. Rather, we are an institution you can be proud of, an institution where you'll feel good about devoting your time and your talents.At NYU Langone Health, we are committed to supporting our workforce and their loved ones with a comprehensive benefits and wellness package. Our offerings provide a robust support system for any stage of life, whether it's developing your career, starting a family, or saving for retirement. The support employees receive goes beyond a standard benefit offering, where employees have access to financial security benefits, a generous time-off program and employee resources groups for peer support. Additionally, all employees have access to our holistic employee wellness program, which focuses on seven key areas of well-being: physical, mental, nutritional, sleep, social, financial, and preventive care. The benefits and wellness package is designed to allow you to focus on what truly matters. Join us and experience the extensive resources and services designed to enhance your overall quality of life for you and your family. NYU Langone Health is an equal opportunity employer and committed to inclusion in all aspects of recruiting and employment. All qualified individuals are encouraged to apply and will receive consideration. We require applications to be completed online.View Know Your Rights: Workplace discrimination is illegal.NYU Langone Health provides a salary range to comply with the New York state Law on Salary Transparency in Job Advertisements. The salary range for the role is $101,493.51 - $155,000.00 Annually. Actual salaries depend on a variety of factors, including experience, specialty, education, and hospital need. The salary range or contractual rate listed does not include bonuses/incentive, differential pay or other forms of compensation or benefits.To view the Pay Transparency Notice, please click here