Distinguished Engineer, Data Platform

About the RoleCloudZero is growing fast. Our customer base is expanding, the data challenges we're solving are getting more complex, and the platform is scaling to match. As a Distinguished Engineer on the Data Engineering team, you'll own some of the hardest infrastructure problems at CloudZero: shaping the next-generation streaming data platform, the dimensional cost model underlying every attribution decision, the hot/cold storage architecture serving both real-time and historical queries, and the query engine that powers our entire product.This is real platform architecture work at real scale, not a consulting role or a review-and-advise job. You'll define the roadmap, drive the foundational decisions, and be a force multiplier for a talented engineering team — evolving CloudZero from batch-oriented pipelines toward a streaming-first architecture where cost attribution reaches engineers within seconds of a resource being used, not the next morning.This role is ideal for an architect who has built systems like this before, has the scars to prove it, and wants to see their decisions matter in direct and measurable ways for customers and for the business.What You'll DoDefine the Data Platform ArchitectureLead end-to-end technical design for CloudZero's next-generation data platform, from event ingestion and stream processing through hot/cold storage and the query layer to the API surfaceDocument architectural decisions, tradeoffs, and migration strategies with the rigor of an RFC-driven processShape and drive every layer of the new architecture: event ingestion, stream processing and enrichment, real-time serving, analytical storage, query layer, and APIDrive Streaming Infrastructure to ProductionDesign and deliver CloudZero's real-time data pipeline from ingestion through enrichment to servingEstablish SLOs for throughput, latency, and correctness, and build the operational playbooks that make this system trustworthy enough to replace the batch pipelines our entire product currently depends onTackle real-time streaming at scale across thousands of customers simultaneously, with fault tolerance, backpressure awareness, and correctness as non-negotiablesTackle the Dimension Cardinality ProblemRedesign CloudZero's dimensional cost model to support high-cardinality, multi-dimensional cost attribution without runaway materialization costsDrive incremental, delta-based materialization strategies using modern open table formats, dramatically reducing expensive full-rebuild jobs and unlocking millions in annual infrastructure savingsEvolve the Query LayerAssess CloudZero's current query infrastructure, drive in-flight migrations to completion, and lead the evolution of the query engine layer going forwardOwn performance optimization across partition pruning, predicate pushdown, and query planning, and set the vision for how the query layer grows as data volumes scale 10xExtend Cost Attribution to Real-TimeEvolve CloudZero's proprietary cost attribution engine from a batch-oriented model to one that assigns complex cost dimensions by team, feature, and customer within seconds of resource usageRethink enrichment, data lineage, and correctness guarantees in a streaming contextShape the Data Engineering RoadmapPartner with product, infrastructure, and analytics engineering to define a multi-year data platform roadmapBuild consensus across engineering leadership on foundational investments including table formats, streaming frameworks, query engines, and schema managementElevate the Engineering TeamParticipate in architecture reviews, contribute to design patterns and best practices, and mentor senior and staff engineers through code review, pairing, and structured feedbackMake everyone around you better, not by directing, but by raising the collective craftWhat You BringData Platform & Architecture10+ years in data engineering with a clear trajectory toward principal or staff-level architectureBuilt and operated large-scale data platforms serving tens of millions of events per day in productionDeep experience with streaming systems such as Kafka, Kinesis, Flink, or Spark Streaming at real production throughputStrong hands-on fluency with modern open table formats including Apache Iceberg, Delta Lake, and Hudi, including compaction, partitioning strategy, and time-travel queriesDesigned hot/cold storage architectures with explicit latency SLOs per tierProven ability to drive a data platform end to end, not just a single layerData Modeling & Dimensional DesignExpert in dimensional data modeling including fact/dimension schema design, slowly changing dimensions, and cardinality managementDeep understanding of the materialization tradeoff space: full vs. incremental, push vs. pull, pre-aggregate vs. query-timeExperience with cost attribution, showback/chargeback, or multi-tenant data partitioning patternsStrong SQL and query optimization background across predicate pushdown, partition pruning, and cost-based query planningQuery Engines & ComputeHands-on with distributed query engines such as Trino, Presto, Spark SQL, or DuckDB including configuration, optimization, and production operationsUnderstands catalog and metadata management and how it couples to query enginesComfortable with cloud data warehouses such as Snowflake, BigQuery, and Redshift and how they integrate with open table formatsExperience driving query engine migrations while maintaining production SLAsEngineering LeadershipTrack record as a technical anchor for a data platform or data engineering teamWrites clear ADRs, RFCs, and technical design docs that bring engineers alongCan drive multi-month, multi-team technical initiatives from inception to production without heavy process overheadCommunicates complex tradeoffs to non-technical stakeholders including product and business leadershipComfortable in a high-autonomy environment: builds consensus, influences through expertise, and helps teams move forwardBonus If You Have...FinOps or cloud cost domain experienceMulti-cloud data ingestion across AWS, Azure, and GCPApache Flink at production scaleLakehouse architecture patternsReal-time feature engineering for MLData mesh or domain-oriented design patternsPrior startup or high-growth SaaS experienceOpen source contributions to the data ecosystemAbout CloudZeroCloudZero is the AI ROI Company. We built the financial control plane for AI: the system finance, IT, and engineering use to connect every AI dollar to the outcome it produced. Across every provider. In real time.AI spend is the fastest-growing line on enterprise P&Ls and the least understood. Only 14% of CFOs can prove AI ROI today. CloudZero answers the question no one else can: what did it cost to produce this outcome, for this customer, on this model.The largest cloud spenders on the planet already run on CloudZero, including Coinbase, Duolingo, DoorDash, and Shutterstock. We processed 14 trillion billing events in the last twelve months. We're the first listed partner on Anthropic's cost and usage API. We've raised over $119 million, including a$56 million Series C backed by leading venture capital firms.Why Join Our Team? At CloudZero, you’ll find a collaborative, fast-moving environment where your work makes a direct impact. We’re a team that values ownership, creativity, and curiosity — and we’re tackling some of the most complex challenges in the cloud space. If you’re excited by working with cutting-edge technology, driving meaningful outcomes, and growing with a company that’s scaling fast, we’d love to hear from you!Compensation Range: $275K - $330KLocationSan Francisco; BostonAddressSan Francisco, CaliforniaEmployment TypeFull timeLocation TypeRemoteDepartmentEngineeringData Engineering & AICompensation$275K – $330K • Offers Equity • Offers Bonus