Lead Data Engineer, Data Platform
CrewAI
About CrewAI CrewAI is the leading framework and enterprise platform for building and orchestrating multi-agent AI systems, powering 300M+ agent executions per month across thousands of companies. As the product, platform, and customer base scale, data is becoming one of the most important systems in the company: how we understand usage, reliability, activation, customer health, cost, governance, and where to invest next. Today, we have meaningful data already, but it is spread across product telemetry, trace data, application databases, analytics tables, Cube models, Metabase dashboards, and team-specific queries. We need someone to turn that into a coherent, trusted, useful data foundation.
The Role You'll be CrewAI's first dedicated data engineering hire. Your job is to own the data foundation end to end: rationalize what exists, improve the infrastructure, define trusted metrics, close instrumentation gaps, and make data accessible enough that product, growth, engineering, customer success, and leadership can actually use it. This is a foundational role with real range. The center of gravity is data infrastructure and analytics engineering: pipelines, warehouse/lake design, semantic modeling, metric definitions, data quality, and self-serve access. You'll also be the person who turns messy questions into clear analysis, reliable dashboards, and better product decisions. This is not a maintenance role. It is a "make data legible and useful for the company" role.
What You'll Do
What We're Looking For
The Role You'll be CrewAI's first dedicated data engineering hire. Your job is to own the data foundation end to end: rationalize what exists, improve the infrastructure, define trusted metrics, close instrumentation gaps, and make data accessible enough that product, growth, engineering, customer success, and leadership can actually use it. This is a foundational role with real range. The center of gravity is data infrastructure and analytics engineering: pipelines, warehouse/lake design, semantic modeling, metric definitions, data quality, and self-serve access. You'll also be the person who turns messy questions into clear analysis, reliable dashboards, and better product decisions. This is not a maintenance role. It is a "make data legible and useful for the company" role.
What You'll Do
- Own and evolve CrewAI's data platform across ingestion, transformation, storage, semantic modeling, BI, and operational data quality.
- Rationalize the existing data estate: product events, execution telemetry, OpenTelemetry-derived traces, application tables, Cube models, Redshift/data-lake tables, Metabase dashboards, and team-specific reporting.
- Establish trusted source-of-truth metrics for the business and product, including executions, active builders/users, activation, deployment health, token and cost usage, customer health, governance adoption, retention, and feature usage.
- Build and maintain the models, pipelines, and metric layers that make those numbers consistent across teams.
- Partner with product and engineering to improve instrumentation, event taxonomy, data contracts, and telemetry coverage for new features.
- Make data self-serve through clear dashboards, documented datasets, reusable metric definitions, and sensible access patterns.
- Improve reliability and trust in the stack through data quality checks, freshness monitoring, lineage, alerting, backfills, and incident/debug workflows.
- Partner with Discovery, product, and go-to-market teams on analysis behind recommendations, customer signals, usage patterns, and roadmap decisions.
- Keep the stack secure and cost-aware, including access control, PII handling, retention, and warehouse/query efficiency.
- Help define how CrewAI uses data internally as the company scales.
What We're Looking For
- Strong data engineering or analytics engineering experience, especially building data foundations in fast-moving product companies.
- Excellent SQL and data modeling skills, with experience designing reliable datasets, fact/dimension models, and metric definitions.
- Experience operating a warehouse or analytics store such as Redshift, Snowflake, BigQuery, Postgres, or similar.
- Familiarity with transformation and modeling tools such as dbt, Cube, semantic layers, or equivalent systems.
- Experience with event pipelines, product telemetry, application data, and BI tools such as Metabase, Looker, Mode, or similar.
- Strong Python for data work, automation, validation, and operational workflows.
- Product sense: you can turn ambiguous questions into useful metrics, and you care whether the numbers are understood correctly.
- Pragmatism: you are comfortable inheriting messy systems, improving them incrementally, and choosing boring reliable solutions when they are right.
- Strong communication and documentation habits. You make data easier for other people to use.
- Comfort being the first dedicated owner in an early-stage, high-growth environment.
- Experience with LLM, agent, observability, trace, usage, or cost analytics.
- Experience with OpenTelemetry, high-volume event data, or operational telemetry.
- Experience with experimentation, causal analysis, activation/retention modeling, or customer health scoring.
- Experience defining event taxonomies and instrumentation standards for SaaS products.
- Familiarity with Rails/Postgres application data, background jobs, and product analytics in B2B SaaS.
- Lightweight ML or recommendation experience, especially where it supports product or customer workflows.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer, Data Platform in San Francisco, CA vacancy
$148.5k - $260.1k
...both a Senior Member of Technical Staff (SMTS) and a Lead Member of Technical Staff (LMTS) to enhance its Enterprise Knowledge Graph platform. Ideal candidates will have significant experience in software engineering, particularly with graph databases and AI technologies...Suggested- ...Position Overview Position: Lead Data Governance Engineer Duration: 12 Months Location: Bay Area, California Work Location: Bay Area, California... ...implementation and adoption of enterprise data governance platforms such as Ataccama, Collibra, or similar tools. Facilitate...SuggestedLocal area
- ...Position Title We are seeking a Lead Data Engineer to architect, build, and lead the development of scalable, cloud‑based data platforms that support enterprise analytics, operational reporting, and advanced data use cases. This role provides technical leadership in designing...Suggested
$112.5k - $149.97k
...Annual Salary Range Lead IT Analyst: $112,497 - $149,972 FLSA Exempt / Union Represented... ...Technology is seeking a Lead Data Engineer to join our dynamic team at the State Bar... ...technical authority for the Bar’s data platform, you will lead the development of scalable...SuggestedWork at officeRemote work- ...build intelligent software that automates media buying, making big data actionable for marketers and connects them to their next customer. Based in San Francisco, we are seeking a talented Lead Data Engineer. RadiumOne generates first‑party data about actual customers —...Suggested
$149.3k - $200.2k
...come. Reach & Scale: The products and platforms this group builds and operates delight... ...sports, entertainment & news. The Product & Data Engineering team is responsible for end to end... ..., messaging and data. Job Summary As a Lead Data Engineer in the Data Platforms team...Work experience placementWorldwide$215.2k - $245.6k
...Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast‑paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you’ll be part of a big group of makers, breakers...InternshipLocal area- ...work on, we'd like to talk. Your Team The data team at Nuna is an interdisciplinary group... ...actuarial science, and research. Within it, data engineering is the technical backbone which creates the architecture, platform, and operations that everything else is built...
$215.2k - $245.6k
## Lead Data EngineerApplylocations: San Francisco, CAtime type: Full timeposted on: Posted... ...customer needs. We are seeking Data Engineers who are passionate about marrying data with... ...building and maintaining the foundational data platforms that empower Capital One to understand...Full timePart timeInternshipH1bLocal area- Salesforce, Inc. is seeking a Data Engineer for its Enterprise Knowledge Graph & AI team in San Francisco, California. The role involves developing and optimizing core knowledge graph components and AI frameworks, focusing on innovation and engineering excellence. The ideal...Flexible hours
- Role: Lead Data Engineer Location: San Francisco, CA (1-2 days a week onsite is must) Job Type: W2 Contract Contract Length: 9 months with... ...design patterns, and building scalable BI solutions Knowledge of cloud platforms (Azure, AWS or equivalent) #J-18808-Ljbffr SaranshContract work2 days per week1 day per week
- ...a global leader in enterprise-grade data analytics and AI solutions, committed... ...with EQT. Website: Job Role Lead, Data Science & Data Engineering - Function: Analytics & AI Consulting... ...designing data pipelines and feature platforms, and translating commercial questions...Worldwide
$200k
...company in San Francisco seeks a Developer Experience Lead to enhance internal tools for their pre-training data team. This role involves leading tooling efforts... .... Candidates should have strong software engineering fundamentals and experience with CI systems. Annual...$215.2k - $245.6k
Capital One National Association is seeking a Lead Data Engineer in San Francisco, CA. The role involves driving technological transformations and collaborating with agile teams to build technical solutions using programming languages like Python, SQL, and Scala. Ideal...$150k - $220k
Mochi Health, based in San Francisco, is seeking a Data Manager to lead the Data team. The role involves executing analytics and data engineering projects while tracking important KPIs. Candidates should have strong SQL and Python skills and experience managing technical...- Tech Lead, Data & Inference Engineer Location: San Francisco Work type: Full Time Compensation: above market base + bonus + equity Roles & Responsibilities... ...design, development and scaling of an end‑to‑end data platform from ingestion to insights, ensuring that data is fast,...Full time
- ...At Sierra, we’re creating a platform to help businesses build better... ...You’ll join a full-stack data team building the systems making... ...of these areas: Platform Engineering You have designed, built,... ...ranking). A track record of leading complex technical projects or...Full timeFlexible hours
- ...APIs. We work across research, engineering, product, policy, safety, and... ...safely. The Trust & Safety Data Engineering team builds the... ...Science, Engineering, and Data Platform to create reliable, privacy-... ...Role We are hiring a Technical Lead Manager to lead and grow the...Relocation package
$179.5k - $221.5k
...understands itself as a business, working closely with Data Science and Analytics. What's unusual here: the platform you're instrumenting is the same one your... ...partner closely with product analytics, product engineering, and data infrastructure to turn business questions...Remote workShift work$130.9k - $220k
...Senior Software Engineer - Data Platform Remote– US Samsara’s Mission Improve the safety, efficiency, and sustainability of the operations that power the global economy. About the Role Samsara is looking for a Senior Software Engineer to join our Data Platform team, which...Full timeRemote workRelocation packageFlexible hours- ...Purpose of Onyx The Onyx Research Data Tech organization is GSK’s research data ecosystem that brings... ...of product and portfolio leadership, data engineering, infrastructure and DevOps, data/metadata/knowledge platforms, and AI/ML and analysis platforms, all geared toward...Work experience placementLocal area
- ...professionals for this role. About us At Sierra, we’re creating a platform to help businesses build better, more human customer experiences... ...customers trust Sierra with their most sensitive workflows and data. You’ll help set and uphold a high bar for security, compliance...Full timeFlexible hours
- ...Data/ETL Engineer (Founding Team) Location: San Francisco Bay Area Type: Full-Time Compensation: Competitive salary + early-stage equity... ...problems. About the Role We’re building a multi-tenant, AI-native platform where enterprise data becomes actionable through semantic...Full time
$60 - $85 per hour
...A fast-growing AI company''s Finance organization is looking for a Data Engineer to develop and maintain the data infrastructure that supports its CFO team. This role offers the opportunity to join a small, dynamic team with a wide-ranging mandate: build data pipelines...Hourly payFull timeContract work- ...San Francisco, CA - Hybrid Responsibilities Strong experience in data engineering, data platforms, distributed systems, or enterprise data infrastructure. Practical experience building AI-enabled data systems, retrieval systems, semantic layers, or data agents. Strong...
- ...Plaid is seeking a Data Engineer in San Francisco to enhance our data systems and build datasets that empower business decisions. This role involves collaboration across teams while advocating for data quality and effective strategies. The ideal candidate should have over...
- ...Senior Data Engineer Location: Bay Area (San Francisco Peninsula) or NYC (secondary) The Opportunity... ..., so you get to design the data platform from scratch instead of untangling years... ...and sustain them. Independent Leader: You lead complex technical initiatives on your own...Work at officeRelocation packageFlexible hours2 days per week
- ...staffing services. Quantix ProTech was recently featured in US News and World Report and Forbes. Job Description Job Title: Data Platform Engineer Location: San Francisco, CA Type: Direct Hire Job Description: Our client in the San Francisco, CA area is looking for a...
- ...combination of inventive research, design, and engineering. Our organization is very flat, and our... ...code. About the role Take data foundations from 0-1 on the bleeding edge... ...data. As one of Cursor’s first Data Platform Engineers, you’ll own the systems that make...Full timeImmediate start
$203k - $282k
...You will collaborate with Data Scientists, Machine Learning Engineers, and Product Engineers to instrument and quantify our ML tools. Through this role you will have frequent opportunities to build the core data pipelines that enable recommendations to the organizational...Full timeWork at officeRemote workHome officeFlexible hours
Do you want to receive more vacancies?
Subscribe and receive similar vacancies to Lead Data Engineer, Data Platform. Be the first to apply!
Related searches
- data engineer analytics San Francisco, CA
- data engineer manager San Francisco, CA
- remote data engineer San Francisco, CA
- junior data engineer remote San Francisco, CA
- senior cloud data engineer San Francisco, CA
- data platform engineer San Francisco, CA
- data infrastructure engineer San Francisco, CA
- aws data engineer San Francisco, CA
- director data engineering San Francisco, CA
- big data developer San Francisco, CA



