Databricks Architect

Job Title: Databricks Architect
Location: St. louis, MO/ Arlington, VA/ NYC, NY & NJ ( Arlington & NY/NJ is Hybrid)
Hire Type: Full time

JOB DESCRIPTION

Primary Architect Databricks Lakehouse & Data Governance

Business Hours
Core U.S. business hours; overlap with EU stakeholders as needed

Role Type
Front-facing techno-functional Databricks architecture & data governance leadership

Role Overview
We are seeking a highly experienced Databricks Architect / Primary Architect Databricks Lakehouse & Data Governance to lead the architecture, governance, security, and modernization of enterprise data platforms on Databricks. The role will serve as the architectural authority for Databricks initiatives from requirement intake and solution design through implementation and production support, partnering with business stakeholders, data owners, governance bodies, security, application teams, data engineering teams, source-system SMEs, vendors, and leadership.

Day to Day Job Duties

• Lead enterprise Databricks Lakehouse architecture using Medallion Architecture (Bronze, Silver, Gold), establishing scalable, reusable, secure, and governed platform patterns.
• Define the target-state Databricks architecture across ingestion, transformation, storage, orchestration, analytics, streaming, data sharing, and consumption layers.
• Design and govern Delta Lake / Delta-based data architectures, including data layout, performance optimization, schema evolution, data lifecycle, and workload-management standards.
• Own the enterprise data-governance architecture on Databricks, with strong focus on Unity Catalog for centralized cataloging, access control, data discovery, lineage, ownership, and governed data sharing.
• Define data domains, catalogs, schemas, ownership models, classification standards, business metadata, technical metadata, and stewardship patterns in partnership with enterprise data-governance teams.
• Establish fine-grained security and access-control patterns using RBAC/ABAC principles, IAM integration, least-privilege access, row/column controls, masking, encryption, secure sharing, and auditability.
• Architect enterprise data-quality controls and observability, including validation rules, quality metrics, reconciliation, freshness/completeness monitoring, lineage, exception management, and SLA/SLO reporting.
• Design high-performance batch and streaming pipelines using Databricks capabilities and distributed processing patterns; establish reusable frameworks, engineering standards, and orchestration practices.
• Provide architecture for governed data exposure through APIs, Delta Sharing / secure data sharing, semantic layers, BI tools, ML/AI consumers, and downstream analytical applications.
• Translate business and governance requirements into conceptual, logical, and physical data models; define modeling standards for lakehouse, dimensional, and enterprise data structures.
• Serve as the first architecture point of contact for new projects and enhancements; lead discovery, feasibility assessment, architecture reviews, requirements clarification, and solution alignment.
• Partner with Information Security, Risk, Compliance, Privacy, and Data Governance teams to embed regulatory and enterprise controls into the Databricks platform and data-product lifecycle.
• Drive migration and modernization of legacy data platforms and workloads to Databricks, including architecture assessment, workload rationalization, migration patterns, performance tuning, and cutover strategy.
• Support production issue analysis and triage during onshore hours, helping resolve data-quality, security, performance, pipeline, and platform issues while minimizing business impact.
• Mentor data engineers and architects, provide design oversight and architecture governance, and work with vendors/system integrators to ensure implementation compliance.

Basic Qualifications

• Minimum 10+ years of experience in Data Architecture, Data Engineering, or enterprise data-platform delivery, including architecture leadership for large-scale data ecosystems.
• Minimum 5+ years of strong hands-on experience with Databricks and modern lakehouse architecture, including production-scale implementation and optimization.
• Deep expertise in Medallion Architecture (Bronze, Silver, Gold), Delta Lake, distributed data processing, Spark/PySpark, SQL, batch processing, and streaming architecture.
• Strong experience with Databricks Unity Catalog and enterprise data-governance capabilities including cataloging, metadata management, lineage, data classification, ownership, access governance, and audit controls.
• Minimum 5+ years of data-governance experience covering data quality, metadata, lineage, stewardship, data ownership, policy enforcement, privacy, security, and compliance controls.
• Strong cloud architecture experience with AWS, Azure, and/or GCP, including IAM, networking, encryption, secrets, storage, security controls, and integration with Databricks.
• Experience designing secure data-consumption patterns including APIs, data sharing, semantic layers, BI integration, and controlled self-service analytics.
• Strong knowledge of conceptual, logical, and physical data modeling; experience with dimensional modeling and enterprise data-modeling standards.
• Strong stakeholder-management skills with the ability to communicate architecture to business, engineering, governance, security, and executive audiences.
• Demonstrated ability to influence cross-functional teams, lead architecture decisions, and operate as a front-facing architect during core U.S. business hours.
• Ability to work onshore in Atlanta with a minimum of 3 days per week in the office.
• Travel: As required based on project and client needs; primary work location is Atlanta, Georgia.
• Degree: Bachelor's degree in Computer Science, Information Systems, Data Engineering, or a related field, or equivalent work experience.

Nice to Have (But Not a Must)

• 12+ years of experience in Data Architecture and/or Enterprise Data Engineering, including large-scale cloud data transformation programs.
• Databricks Certified Data Engineer Professional and/or Databricks Certified Data Architect / relevant advanced Databricks certification.
• Experience implementing enterprise governance using Unity Catalog across multiple workspaces, business domains, and cloud environments.
• Experience with governance/catalog platforms such as Collibra, Alation, Microsoft Purview, or equivalent and integration with Databricks governance patterns.
• Cloud Architect certification in AWS, Azure, and/or GCP.
• TOGAF or equivalent architecture certification.
• Experience in regulated industries such as banking, financial services, insurance, healthcare, or other highly controlled environments.

Leadership & Behavioral Competencies

• Strong architectural thinking and end-to-end systems-design capability.
• Excellent stakeholder communication, workshop facilitation, and executive presentation skills.
• Strong governance mindset with the ability to balance data democratization, security, control, and delivery speed.
• Strategic mindset with strong execution orientation and ability to convert standards into implementable engineering patterns.
• Ability to influence cross-functional teams and drive architecture and governance decisions.
• Mentorship and team-development experience.

Success Criteria

• Establish a scalable, secure, high-performing Databricks Lakehouse architecture with standardized Bronze, Silver, and Gold patterns.
• Implement a robust Unity Catalog-based governance model covering ownership, access, cataloging, classification, lineage, auditability, and secure sharing.
• Drive adoption of standardized data-quality, metadata, lineage, security, privacy, and compliance practices across data products.
• Establish governed and performant data-exposure frameworks for analytics, BI, AI/ML, APIs, and downstream consumers.
• Improve platform reuse, engineering productivity, workload performance, data trust, and time-to-delivery through architecture standards and reusable patterns.
• Enable effective collaboration with onshore stakeholders and rapid response to production, data-quality, security, and operational issues.

Why Join Us

• Opportunity to shape enterprise Databricks, lakehouse, and data-governance strategy.
• Work on modern cloud-native data platforms and large-scale transformation initiatives.
• Influence enterprise standards for data security, governance, quality, lineage, and democratization.
• High leadership visibility and strategic impact across business and technology teams.