Senior Data Architect - Databricks Platform

Role Description

Architect responsible for designing and hardening enterprise-scale Databricks data platforms within client engagements — building the guardrails, automation, and standards that let a growing number of business teams safely build on the platform without it collapsing into unmanaged "click-ops." Architects maintain primary depth in one core area (Data Platform Development or Data Migration & Modernization) while maintaining architectural awareness across both. Databricks depth is expected regardless of focus area.

Qualifications

• Lead architectural design for Databricks-based data platforms or modernization programs, including workspace consolidation, permissioning/persona models, and configuration-as-code patterns.
• Databrick platform designs align with AWS leading-practice architecture patterns (as we are AWS centric).
• Design of high-volume batch and real-time streaming pipelines leveraging PySpark and Structured Streaming (aka Apache Spark).
• Design and integration of Databrick Unity catalog with third party metadata repositories (e.g., AWS Glue Catalog, Snowflake Horizon).
• Design and implement automation (ConfigOps), CI/CD, and guardrail patterns that let business teams safely extend the platform without compromising governance.
• Build and maintain certified data products, a semantic layer, and lineage/metadata exposure so the platform can support both self-service and, eventually, agent-driven use cases.
• Ensure technical viability, scalability, and production readiness of platform components.
• Communicate architectural decisions and trade-offs clearly to both technical and business stakeholders; produce architectural documentation, assessments, and roadmaps.
• Partner with client business and engineering teams to understand upstream use-case needs and translate them into platform capabilities and guardrails, rather than working from requirements handed down secondhand.

Requirements

• End-to-end modern data platform architecture design (lakehouse, ingestion, processing, access layers).
• Hands-on Databricks experience: workspace/persona architecture, Unity Catalog, lakehouse patterns, job orchestration, and native governance tooling (governed data platform patterns, lineage, access controls and maintaining business, technical and compliance metadata), tune and optimize Spark configurations, clusters and Delta Lake storage.
• Experience architecting scalable cloud-native systems (AWS).
• Advanced distributed processing and streaming architectures.
• Platform security, networking, and automation.
• Cost estimation and performance optimization.
• Leading multi-database migration and modernization initiatives, including onto Databricks.
• Deep expertise in OLTP/OLAP system design and schema evolution.
• Advanced performance tuning, HA/DR, and security patterns.
• Cloud-native database architecture across managed services.
• Migration strategy, cutover planning, and risk mitigation.
• Architecture documentation and standards enforcement.
• IaC and automation strategy (Terraform).
• FinOps-aligned design, including ability to audit and optimize cloud infrastructure expense.
• Ability to lead and facilitate technical discussions between key business, technology and knowledge stakeholders.
• Ability to develop Databrick roadmaps and implementation roadmaps.
• Experience in leading, developing and mentoring delivery teams.

Experience Requirements

• 6+ years in data engineering, database engineering, or architecture roles.
• 3+ years designing cloud-based data solutions.
• 3+ years in the AWS data landscape.
• 3+ years hands-on Databricks experience required.
• High business acumen — comfortable working directly with client business stakeholders to understand their data and priorities, and translating that into governed, well-documented platform capabilities.
• Advanced English (minimum C1).

Benefits

• 100% remote work.
• Medical Insurance for you and eligible dependents.
• Generous holidays and flexible PTO.
• Competitive phantom equity.
• Paid for exams and certifications.
• Peer bonus awards.
• State of the art laptop and tools.
• Equipment & Office Stipend.
• Individual professional development plan.
• Annual stipend for Learning and Development.
• Work with an amazing worldwide team and in an incredible corporate culture.
• This role may require up to 25% travel, depending on business needs.