Databricks - Data Architect

Data Architect - Databricks

Location: Remote US Based

Overview:

We are seeking a Databricks Data Architect to support the design, implementation, and optimization of cloud-native data platforms built on the Databricks Lakehouse architecture. This is a hands-on, engineering-driven role requiring deep technical experience with Apache Spark, Delta Lake, and scalable data pipeline development, combined with early-stage architectural responsibilities.

This role involves close onsite collaboration with client stakeholders , translating analytical and operational requirements into robust, high-performance data architectures while ensuring best practices in data modeling, governance, reliability, and cost efficiency.

Key Responsibilities:

• Design, develop, and maintain batch and near-real-time data pipelines using Databricks, PySpark, and Spark SQL
• Implement Medallion (Bronze/Silver/Gold) Lakehouse architectures , ensuring quality, lineage, and transformation logic across layers
• Build and manage Delta Lake tables , including schema evolution, ACID transactions, time travel, and optimized data layouts
• Apply Spark and Delta performance tuning techniques , including partitioning strategies, Z-Ordering, caching, broadcast joins, and execution tuning
• Support dimensional and analytical data modeling for BI and analytics consumption
• Define and support data ingestion patterns including batch, incremental, CDC, and streaming where applicable
• Troubleshoot and resolve pipeline failures, data quality issues, and Spark performance bottlenecks

Required Skills:

• Experience with Databricks Unity Catalog , governance, and access control
• Experience with Databricks Workflows, Apache Airflow, or Azure Data Factory for orchestration
• Familiarity with Spark Structured Streaming, Kafka, and CDC frameworks
• Experience with data quality and observability frameworks
• Integration experience with Power BI, Tableau, or Looker
• Awareness of cloud cost optimization strategies