Principal Data Engineer

Principal Data Engineer

Most data platform roles ask you to extend something someone else has already built. This one starts with a blank canvas.

As Principal Data Engineer, you'll architect Gather AI's data foundation from the ground up: separating analytical workloads from live production traffic, building a semantic layer so metrics are defined once and stay consistent everywhere, and linking structured records to real drone imagery and video with full traceability. You'll prove the model end to end on Gather's drone product, then generalize it so every new product extends the foundation instead of rebuilding it, all while working as a Principal-level individual contributor with real influence across engineering, product, and leadership.

What You'll Do

• Architect a greenfield, multi-layer data warehouse (raw, refined, serving) that separates analytical workloads from production OLTP traffic.
• Deliver a governed, self-service data-access layer for internal consumers first (Product, CSM, Deployment/Operations, and Leadership) as Phase 1, ahead of customer-facing conversational analytics.
• Build a semantic and metrics layer so every metric, such as "scan accuracy by site," is defined once in code and stays identical across every dashboard and product, making self-service safe from metric drift.
• Own the quality bar: 99%+ availability SLA with freshness guarantees, 100% traceability, zero cross-tenant leakage, 99.5%+ pipeline success, and no data loss.
• Design tenant isolation, per-tenant cost attribution, and schema and row-level RBAC to scale toward hundreds of tenants (300+ target), not today's fleet size.
• Own data-ingestion correctness at the boundary with the integration/backend team, covering data contracts, schema validation, and pipeline quality, so WMS data lands in the right place, shape, and time across WMS versions.
• Stand up a data catalog and lineage layer (Purview as the Azure-native fit, DataHub as the open-source alternative) so every consumer can find data, see ownership, and trace lineage when a metric looks wrong.
• Prove the foundation end to end on Gather's drone product, then generalize it so each new product extends the model instead of rebuilding it
• Act as the connective tissue between product and ML (3DCC, damage detection). Link structured records to unstructured drone imagery and video with full traceability, and stand up the data-infra readiness for feature stores and annotation pipelines on one trusted foundation.

What You'll Need

• 10+ years in data engineering, with 3+ years architecting data platforms for data products, analytics, or AI-driven products.
• Proven experience building a greenfield data warehouse and leading an OLTP to OLAP transition, not just maintaining an existing one.
• Deep expertise designing multi-layer transformation architectures and reusable frameworks that scale across multiple product areas.
• Expert SQL and dbt, hands-on ELT and orchestration, and large-scale or streaming data experience.
• Production experience on a major cloud (Azure preferred, AWS or GCP acceptable), plus infrastructure as code and CI/CD.
• Track record with data quality, security, governance, and multi-tenancy in production environments.
• Data transformation and modeling that turns raw multi-source data into refined, serving-ready datasets (raw to refined to serving).
• Pipeline orchestration and workflow automation for scheduling, dependency management, and reliable execution across data flows.
• Large-scale and distributed processing of high-volume batch data.
• Real-time and streaming ingestion that captures and processes event data as it arrives.
• Semantic and metrics-layer design that defines business metrics once and serves them consistently to every consumer.
• Serving-layer optimization for fast, low-latency consumption through wide and flattened tables and pre-computed metrics.
• Cloud data engineering and infrastructure automation that provisions, deploys, and operates the platform reproducibly (cloud-native, infrastructure as code, CI/CD).
• Data quality, observability, and lineage that ensure trust, freshness, and end-to-end traceability.
• Security, governance, and multi-tenancy including tenant isolation, access control, and resiliency.
• Multimodal data integration that links structured records to unstructured image and video (drone captures) with traceability.

Ways of Working

• Treats data as a product for internal consumers, not just a pipeline feeding dashboards.
• Comfortable making long-lead architecture calls (platform, isolation model) with incomplete consensus.
• Strong cross-functional collaborator, works closely with integration/backend, ML, product, customer success teams and internal analytics consumers.

Nice to Have

• Experience modeling structured data linked to unstructured or blob data such as images, video, or sensor files
• Experience with feature stores, annotation pipelines, or ML data infrastructure supporting computer vision products.
• IoT, edge, or device-telemetry background
• BI or presentation-layer and dashboard design experience
• Warehousing, logistics, or supply-chain domain knowledge