Technical Lead - Databricks Developer

Role Technical Lead - Databricks Developer

Location Remote

Client BeOne Medicine

Position Overview

We are partnering with a world-class healthcare and biotechnology leader to find a highly skilled Technical Lead - Databricks Developer . In this critical role, you will design and implement robust, scalable data engineering solutions using the Databricks Lakehouse architecture.

This position is ideal for a hands-on technical leader who excels at bridging the gap between complex data processing and client business needs. You will collaborate closely with enterprise architects and lead engineers, taking ownership of end-to-end data pipelines, optimizing system performance, and driving modern data governance practices in a dynamic corporate environment.

Key Responsibilities

• Architecture & Solution Design: Collaborate with architects and lead engineers to design and build solutions that meet both functional and non-functional enterprise requirements.
• Pipeline Engineering: Architect, build, and manage complex, scalable data pipelines supporting both batch and streaming data processing.
• Databricks Orchestration & Governance: Implement and manage Databricks workflows orchestration, leveraging Unity Catalog for data governance and access control.
• Performance Optimization: Proactively identify bottlenecks and drive rigorous performance optimization initiatives to maximize efficiency and significantly reduce compute costs.
• Client Engagement: Work directly with customers and stakeholders, utilizing exceptional communication skills to translate business challenges into technical deliverables.

Required Qualifications

• Databricks Expertise: Must have successfully implemented at least one enterprise project end-to-end within Databricks. Deep understanding of the Databricks Lakehouse concept and its implementation in large-scale environments is mandatory.
• Core Data Components: Proven, hands-on experience with core ecosystem components including Delta Lake, Unity Catalog, Databricks Workflows, and basic data governance principles.
• Programming & Frameworks: Excellent, production-level coding skills in Python, strong proficiency in SQL and Spark-SQL, and extensive knowledge of Apache Spark and Hive data processing frameworks.
• Technical Foundations: Strong foundational knowledge of data structures, algorithms, and advanced analytical problem-solving skills.
• Communication Skills: Exceptional verbal and written communication skills with the proven ability to engage confidently and directly with enterprise clients.

Preferred Qualifications

• Industry Trends: Demonstrated knowledge of relevant industry standards, emerging data engineering trends, and modern cloud ecosystems.
• Continuous Growth: A strong, proactive attitude toward learning new skills, upskilling in cutting-edge data technologies, and mentoring peers.

Education

• Bachelors or Masters in Information Technology, Computer Science or relevant field.