Principal Consultant - Databricks

Job Description

Job Description

OZ - Principal Consultant - Databricks

We believe work should be innately rewarding and a team-building venture. Working with our teammates and clients should be an enjoyable journey where we can learn, grow as professionals, and achieve amazing results. Our core values revolve around this philosophy. We are relentlessly committed to helping our clients achieve their business goals, leapfrog the competition, and become leaders in their industry. What drives us forward is the culture of creativity combined with a disciplined approach, passion for learning & innovation, and a ‘can-do’ attitude!

What We're Looking For:

We're seeking a highly experienced Principal Consultant - Databricks to serve as the senior technical authority for complex enterprise data-platform engagements. The ideal candidate combines deep Azure Databricks and Lakehouse expertise with strong consulting, architecture, delivery leadership, and executive communication skills.

Position Overview:

As a Principal Consultant - Databricks, you will own the technical vision, target-state architecture, standards, and roadmaps for Azure Databricks engagements from discovery through production delivery. You will shape secure, scalable Lakehouse solutions, guide multidisciplinary teams, support pre-sales and delivery planning, and ensure that platform decisions align with customer outcomes, operational requirements, security, performance, reliability, and cost goals. This is a remote U.S.-based role working primarily in the Eastern Time Zone.

Key Responsibilities:

• Own Technical Vision and Architecture: Define target-state architecture, principles, standards, and roadmaps for complex Azure Databricks engagements.
• Lead Discovery and Architecture Workshops: Translate business priorities, security needs, operational constraints, and success measures into actionable solution designs.
• Design Enterprise Lakehouse Platforms: Architect secure and scalable solutions spanning ingestion, storage, transformation, orchestration, analytics, data sharing, and AI/ML enablement.
• Establish Platform Standards: Define engineering, governance, identity, networking, CI/CD, observability, reliability, performance, and cost-optimization practices.
• Provide Delivery Oversight: Review critical implementation artifacts, identify risks early, guide technical decisions, and maintain alignment between sold and delivered solutions.
• Support Pre-sales and Solutioning: Contribute to qualification, solution shaping, estimates, staffing plans, proposals, statements of work, demonstrations, and executive presentations.
• Build Reusable Consulting Assets: Develop reference architectures, accelerators, assessment methods, estimation models, delivery playbooks, and thought leadership.
• Mentor and Collaborate: Guide architects, engineers, and consultants while partnering with Databricks, Microsoft, customer stakeholders, sales teams, and internal practices.

What You're Looking For:

If you’re looking for an opportunity to work in a fast-growing market, surrounded by talented, motivated, and global colleagues who thrive on helping clients meet their most pressing business goals, we are the company for you. If you’re driven, passionate, and want to be a ’key player’ in a company’s growth, we invite you to make a difference with a company that its employees define. We want you to be bold, take risks, and imagine a better way to work. We should talk if we just described you!

About Us:

With more than 25 years of experience, OZ brings trusted, deep expertise in Azure Cloud Solutions, Power Apps Application Development, Intelligent Automation, Enterprise Application Integration, Azure Data Strategy, and Artificial Intelligence to deliver rapid, effective results.

As a Microsoft Partner, our teams act as trusted advisors, combining technical depth with strong business acumen to solve complex challenges and drive measurable results.

OZ is dedicated to creating a seamless blend between work and life, offering our team the flexibility to work hybrid and/or remotely. We provide competitive pay and a well-rounded benefits package, including full health coverage, a 401K, and unlimited PTO. Our culture is built on collaboration, growth, and balance - you’ll have the tools and support you need to thrive. Join a team that truly lives its values and invests in your success.

Requirements

Requirements:

• 10+ years of experience in enterprise data and analytics consulting, including:

• 5+ years architecting and delivering production Databricks or Lakehouse solutions
• 3+ years in senior technical leadership, solution architecture, or principal-level consulting roles.
• o   Must have 3+ years of consulting experience working with a Property & Casualty (P&C) insurance company. Health insurance experience alone would not meet this requirement.

• Demonstrated experience as the senior technical authority for complex enterprise data-platform engagements, with accountability from discovery through production delivery.
• Deep Azure Databricks expertise across platform architecture, data engineering, Unity Catalog governance, operationalization, performance, and cost management.
• Strong command of Apache Spark, PySpark, Python, SQL, Delta Lake, batch and streaming patterns, ETL/ELT, data modeling, data quality, performance tuning, and production troubleshooting.
• Working knowledge of Microsoft Entra ID, Azure networking and private connectivity, ADLS, secrets and key management, monitoring, and related Azure data services.
• Experience with Git-based development, automated testing, CI/CD, environment promotion, infrastructure as code, deployment automation, observability, runbooks, and support models.
• Strong consulting and business acumen, with the ability to connect architecture decisions to customer outcomes, delivery risk, scope, margin, adoption, and platform consumption.
• Executive-level written, verbal, facilitation, and presentation skills, with the technical depth to engage architects and engineers.
• Experience in discovery, estimation, staffing, proposal or SOW development, technical pre-sales, delivery leadership, and multidisciplinary team guidance.
• Self-directed, collaborative leadership with sound judgment, resilience, empathy, and the ability to create structure in ambiguous situations.

Preferred Qualifications:

• Databricks Certified Data Engineer Professional strongly preferred.
• Azure Platform Architect Badge and/or Databricks Data + AI Platform Administrator accreditation preferred.
• Databricks Partner Champion or Delivery Expert recognition, partner or Center of Excellence leadership, co-sell experience, or work in regulated and security-sensitive industries.
• Candidates without an appropriate Databricks credential must be willing to complete an OZ-approved, role-aligned certification or accreditation path after joining.
• Databricks Certified Generative AI Engineer Associate or Machine Learning Professional is a plus.
• Experience with large-scale platform modernization, cloud transformation, data-warehouse migration, or consolidation involving Databricks.
• Experience creating reusable offerings, accelerators, reference architectures, assessment frameworks, delivery playbooks, or intellectual property.
• Experience overseeing production analytics, machine-learning, or AI workloads beyond proof-of-concept delivery.

Tools & Technologies:

• Azure Databricks, Delta Lake, Unity Catalog, Databricks SQL, Lakeflow Jobs and pipelines, Apache Spark, PySpark, Python, and SQL.
• Microsoft Azure, Microsoft Entra ID, Azure Storage/ADLS, Azure networking, Private Link, Key Vault, and Azure Monitor.
• Git, GitHub and/or Azure DevOps, automated testing, CI/CD, Terraform or comparable infrastructure as code, Databricks CLI and APIs, and Declarative Automation Bundles.
• MLflow, model serving, Power BI, Microsoft Fabric, Azure Data Factory, and Microsoft Purview exposure preferred.

Competencies:

• Databricks and Lakehouse architecture leadership
• Enterprise data-platform design and governance
• Consulting experience, Property & Casualty (P&C) Insurance
• Technical pre-sales and solution shaping
• Delivery oversight and risk management
• Performance, reliability, security, and cost optimization
• Executive stakeholder communication and facilitation
• Team mentorship and cross-functional collaboration

Benefits

• Competitive Pay
• 401K
• Health Coverage