Principal Data Engineer

Role Description

We're hiring a Principal Data Engineer who wants to stay deeply technical while growing into people leadership. This is fundamentally an individual-contributor / architect role and you'll be the primary hands-on builder of our data platform, writing production code in Databricks and Azure on a daily basis. Alongside that, you'll manage and mentor a small team of 3–5 offshore data engineers, giving you a natural runway toward a full-time Data Engineering Manager role as the team and platform scale.

You'll be the technical owner of a global data platform that unifies data across 10+ products, supports both transactional and analytical workloads, and is architected from day one to onboard future acquisitions cleanly. You'll also design the platform's orchestration layer to support complex agentic AI workflows, including agent-to-agent (A2A) communication between autonomous systems.

What You'll Do

• Platform Architecture & Hands-On Engineering:
  • Architect and build a unified, multi-tenant data platform on Databricks and Azure that integrates data from 10+ product lines into a single source of truth.
  • Design the platform to be growth-ready, with modular ingestion patterns, flexible schema onboarding, and governance structures.
  • Build and optimize pipelines that serve both transactional and analytical use cases using the medallion architecture.
  • Design and implement the orchestration layer for agentic AI workflows, including multi-step task automation.
  • Write and review production-grade PySpark, SQL, and Python code.
  • Own performance, scalability, and cost optimization across the Databricks/Azure environment.
• Team Leadership (Growing Over Time):
  • Directly manage and mentor 3–5 offshore data engineers, setting technical direction and reviewing their code.
  • Establish engineering standards, CI/CD practices, and testing/observability norms for the team.
  • As the platform and team grow, take on expanded management scope, with a clear path toward a Data Engineering Manager.
• Cross-Functional & Strategic:
  • Partner with Product, Engineering, Security, and M&A/Corp Dev stakeholders to ensure the data platform's architecture supports business growth.
  • Establish data governance, lineage, access control, and compliance practices across a growing, multi-product data estate.
  • Evaluate and integrate new tools to keep the platform modern and extensible.

Qualifications

• 8+ years of experience in data engineering or data architecture, with deep, hands-on experience.
• Expert-level, hands-on experience with Databricks.
• Strong hands-on experience with the Azure data ecosystem.
• Expert-level SQL and Python (PySpark); comfortable debugging Spark jobs at a deep level.
• Experience designing data platforms that serve both transactional and analytical workloads.
• Familiarity with agentic AI orchestration concepts and frameworks.
• Prior experience leading or mentoring engineers with genuine interest in growing into a people-management role.
• Experience working with offshore/distributed engineering teams.
• Solid understanding of data modeling, medallion architecture, CI/CD for data workflows, and Infrastructure-as-Code.
• Excellent communication skills; able to work directly with cross-functional stakeholders.

Requirements

• Databricks Certified Data Engineer Professional or Azure Data Engineer Associate/Expert certification (nice to have).
• Experience integrating a data platform after a merger or acquisition.
• Experience with vector databases and RAG pipelines.
• Experience with Kafka, Event Hubs, or other streaming technologies.
• Background in a regulated industry with associated compliance requirements.

Benefits

• Fully remote work, based anywhere in the U.S.
• A high-ownership architect role with a clear, structured path to engineering management.
• Competitive salary + equity/bonus (commensurate with experience and location).
• Comprehensive health, dental, and vision coverage.
• 401(k) with company match.
• Flexible PTO.
• Home office / equipment stipend.
• Budget for conferences, certifications, and continued learning.