Principal Data Solution Architect (Databricks)

Overview We are seeking a Principal Data Solution Architect to serve as the senior‑most technical authority for end‑to‑end data architectures, cloud data platforms, integration strategies, and enterprise‑scale modernization initiatives. This role operates at the intersection of architecture, engineering, analytics, AI, and mission strategy. The Principal Data Solution Architect shapes the vision for how data is collected, governed, transformed, stored, accessed, and activated across complex environments—ultimately enabling reliable analytics, ML/AI capabilities, and mission‑critical applications. This leader drives architectural strategy across data platforms, lakehouses, warehouses, integration layers, and AI/ML pipelines—leveraging modern platforms such as Databricks to enable unified analytics, governance, and AI‑ready data ecosystems. Contributions Serve as the chief architect for data platform modernization, designing enterprise data ecosystems including lakehouse architectures (e.g., Databricks Lakehouse), data mesh/fabric patterns, domain‑driven design, and multi‑cloud data strategies. Translate mission and business needs into actionable data architecture roadmaps, reference architectures, and solution blueprints. Architect and operationalize Databricks environments, including workspace design, Unity Catalog governance, Delta Lake architecture, cluster and job orchestration, and secure multi‑tenant data access patterns. Design and implement scalable ingestion, transformation, and data serving layers using batch, streaming, CDC, API‑based, and event‑driven patterns (e.g., Delta Live Tables, Structured Streaming, medallion architecture). Lead enterprise data modeling strategy, including canonical data models, semantic layers, MDM architectures, metadata systems, and AI/ML‑aligned feature modeling. Partner with Data Engineering, Data Science, AI/ML Engineering, and LLMOps/MLOps teams to enable analytics, ML, RAG systems, and advanced automation use cases. Define and enforce enterprise data standards, including schema evolution, data contracts, quality frameworks, lineage, observability, data zoning, and Zero Trust access patterns. Drive platform engineering decisions, including storage optimization (e.g., Delta Lake), cluster sizing, workload isolation, compute orchestration, network design, and cost‑performance optimization within distributed environments such as Databricks. Guide selection and adoption of enterprise data platforms, with deep expertise in Databricks and complementary technologies such as Snowflake, Redshift, Synapse, and BigQuery. Oversee architectural governance, including design reviews, performance evaluations, cloud readiness assessments, and alignment with cybersecurity and compliance requirements. Serve as a senior advisor to client executives, framing modernization strategies, defining investment pathways, and articulating value realization for enterprise data initiatives. Mentor Data Engineers, Data Architects, and cross‑functional technical staff to strengthen architectural maturity across programs. Develop reusable frameworks, architectural patterns, playbooks, and accelerators to improve consistency and reduce delivery timelines. Stay current with emerging trends in cloud‑native data ecosystems, distributed compute, metadata automation, AI‑ready architectures, and federal data regulations. Contribute to the growth of the AI & Data Exploitation Practice through thought leadership and technical innovation. Qualifications Ability to hold a position of public trust. Bachelor’s, Master’s, or Ph.D. in Computer Science, Data Engineering, Information Systems, Cloud Architecture, or a related technical field. 10+ years of experience designing enterprise‑scale data solutions, including significant architectural leadership. Expert knowledge of modern data architectures, including lakehouse, data mesh, data fabric, MDM, event‑driven architectures, and domain‑driven design. Deep experience with cloud‑native data ecosystems across AWS, Azure, or GCP, including storage, compute, orchestration, serverless, containerization, and security services. Mastery of distributed data processing frameworks (e.g., Apache Spark, Databricks, Flink, Kafka) with strong proficiency in SQL and Python. Hands‑on experience designing and implementing lakehouse architectures using Databricks, including Delta Lake, Unity Catalog, and enterprise‑scale governance patterns. Proven ability to design ingestion pipelines, transformation logic, metadata systems, feature stores, and data serving layers optimized for analytics and AI workloads. Strong understanding of enterprise data governance, including cataloging, lineage, privacy, tagging, and Zero Trust access controls. Strong communication skills with the ability to influence senior stakeholders and translate complex concepts into actionable insights. Demonstrated success mentoring technical staff, leading architecture reviews, and guiding multi‑team delivery across complex programs. Experience collaborating with Data Science, AI/ML, and MLOps teams to support scalable model development and operationalization. Preferred certifications (highly relevant): AWS Data Analytics Specialty AWS Solutions Architect – Professional Azure Solutions Architect Expert Databricks Data Engineer Professional Google Professional Data Engineer Snowflake SnowPro Advanced Architect TOGAF or cloud architecture frameworks About steampunk Steampunk relies on several factors to determine salary, including but not limited to geographic location, contractual requirements, education, knowledge, skills, competencies, and experience. The projected compensation range for this position is $165,000 to $210,000. The estimate displayed represents a typical annual salary range for this position. Annual salary is just one aspect of Steampunk’s total compensation package for employees. #J-18808-Ljbffr