Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer, Data Platform

CrewAI

About CrewAI

CrewAI is the leading framework and enterprise platform for building and orchestrating multi-agent AI systems, powering 300M+ agent executions per month across thousands of companies. As the product, platform, and customer base scale, data is becoming one of the most important systems in the company: how we understand usage, reliability, activation, customer health, cost, governance, and where to invest next.

Today, we have meaningful data already, but it is spread across product telemetry, trace data, application databases, analytics tables, Cube models, Metabase dashboards, and team-specific queries. We need someone to turn that into a coherent, trusted, useful data foundation.
The Role

You'll be CrewAI's first dedicated data engineering hire. Your job is to own the data foundation end to end: rationalize what exists, improve the infrastructure, define trusted metrics, close instrumentation gaps, and make data accessible enough that product, growth, engineering, customer success, and leadership can actually use it.

This is a foundational role with real range. The center of gravity is data infrastructure and analytics engineering: pipelines, warehouse/lake design, semantic modeling, metric definitions, data quality, and self-serve access. You'll also be the person who turns messy questions into clear analysis, reliable dashboards, and better product decisions.

This is not a maintenance role. It is a "make data legible and useful for the company" role.
What You'll Do
  • Own and evolve CrewAI's data platform across ingestion, transformation, storage, semantic modeling, BI, and operational data quality.
  • Rationalize the existing data estate: product events, execution telemetry, OpenTelemetry-derived traces, application tables, Cube models, Redshift/data-lake tables, Metabase dashboards, and team-specific reporting.
  • Establish trusted source-of-truth metrics for the business and product, including executions, active builders/users, activation, deployment health, token and cost usage, customer health, governance adoption, retention, and feature usage.
  • Build and maintain the models, pipelines, and metric layers that make those numbers consistent across teams.
  • Partner with product and engineering to improve instrumentation, event taxonomy, data contracts, and telemetry coverage for new features.
  • Make data self-serve through clear dashboards, documented datasets, reusable metric definitions, and sensible access patterns.
  • Improve reliability and trust in the stack through data quality checks, freshness monitoring, lineage, alerting, backfills, and incident/debug workflows.
  • Partner with Discovery, product, and go-to-market teams on analysis behind recommendations, customer signals, usage patterns, and roadmap decisions.
  • Keep the stack secure and cost-aware, including access control, PII handling, retention, and warehouse/query efficiency.
  • Help define how CrewAI uses data internally as the company scales.
Requirements
What We're Looking For
  • Strong data engineering or analytics engineering experience, especially building data foundations in fast-moving product companies.
  • Excellent SQL and data modeling skills, with experience designing reliable datasets, fact/dimension models, and metric definitions.
  • Experience operating a warehouse or analytics store such as Redshift, Snowflake, BigQuery, Postgres, or similar.
  • Familiarity with transformation and modeling tools such as dbt, Cube, semantic layers, or equivalent systems.
  • Experience with event pipelines, product telemetry, application data, and BI tools such as Metabase, Looker, Mode, or similar.
  • Strong Python for data work, automation, validation, and operational workflows.
  • Product sense: you can turn ambiguous questions into useful metrics, and you care whether the numbers are understood correctly.
  • Pragmatism: you are comfortable inheriting messy systems, improving them incrementally, and choosing boring reliable solutions when they are right.
  • Strong communication and documentation habits. You make data easier for other people to use.
  • Comfort being the first dedicated owner in an early-stage, high-growth environment.
Bonus
  • Experience with LLM, agent, observability, trace, usage, or cost analytics.
  • Experience with OpenTelemetry, high-volume event data, or operational telemetry.
  • Experience with experimentation, causal analysis, activation/retention modeling, or customer health scoring.
  • Experience defining event taxonomies and instrumentation standards for SaaS products.
  • Familiarity with Rails/Postgres application data, background jobs, and product analytics in B2B SaaS.
  • Lightweight ML or recommendation experience, especially where it supports product or customer workflows.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer, Data Platform in San Francisco, CA vacancy
  • $148.5k - $260.1k

     ...both a Senior Member of Technical Staff (SMTS) and a Lead Member of Technical Staff (LMTS) to enhance its Enterprise Knowledge Graph platform. Ideal candidates will have significant experience in software engineering, particularly with graph databases and AI technologies... 
    Suggested

    SwiftCruit

    San Francisco, CA
    2 days ago
  •  ...Position Overview Position: Lead Data Governance Engineer Duration: 12 Months Location: Bay Area, California Work Location: Bay Area, California...  ...implementation and adoption of enterprise data governance platforms such as Ataccama, Collibra, or similar tools. Facilitate... 
    Suggested
    Local area

    Caliber Smart

    San Francisco, CA
    4 days ago
  •  ...Position Title We are seeking a Lead Data Engineer to architect, build, and lead the development of scalable, cloud‑based data platforms that support enterprise analytics, operational reporting, and advanced data use cases. This role provides technical leadership in designing... 
    Suggested

    Q-Cells

    San Francisco, CA
    3 days ago
  • $112.5k - $149.97k

     ...Annual Salary Range Lead IT Analyst: $112,497 - $149,972 FLSA Exempt / Union Represented...  ...Technology is seeking a Lead Data Engineer to join our dynamic team at the State Bar...  ...technical authority for the Bar’s data platform, you will lead the development of scalable... 
    Suggested
    Work at office
    Remote work

    State Bar CA

    San Francisco, CA
    5 days ago
  •  ...build intelligent software that automates media buying, making big data actionable for marketers and connects them to their next customer. Based in San Francisco, we are seeking a talented Lead Data Engineer. RadiumOne generates first‑party data about actual customers —... 
    Suggested

    Stepping Up Solutions

    San Francisco, CA
    4 days ago
  • $149.3k - $200.2k

     ...come. Reach & Scale: The products and platforms this group builds and operates delight...  ...sports, entertainment & news. The Product & Data Engineering team is responsible for end to end...  ..., messaging and data. Job Summary As a Lead Data Engineer in the Data Platforms team... 
    Work experience placement
    Worldwide

    The Walt Disney Company

    San Francisco, CA
    2 days ago
  • $215.2k - $245.6k

     ...Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast‑paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you’ll be part of a big group of makers, breakers... 
    Internship
    Local area

    Capital One National Association

    San Francisco, CA
    3 days ago
  •  ...work on, we'd like to talk. Your Team The data team at Nuna is an interdisciplinary group...  ...actuarial science, and research. Within it, data engineering is the technical backbone which creates the architecture, platform, and operations that everything else is built... 

    Nuna Inc

    San Francisco, CA
    3 days ago
  • $215.2k - $245.6k

    ## Lead Data EngineerApplylocations: San Francisco, CAtime type: Full timeposted on: Posted...  ...customer needs. We are seeking Data Engineers who are passionate about marrying data with...  ...building and maintaining the foundational data platforms that empower Capital One to understand... 
    Full time
    Part time
    Internship
    H1b
    Local area

    Capital One

    San Francisco, CA
    2 days ago
  • Salesforce, Inc. is seeking a Data Engineer for its Enterprise Knowledge Graph & AI team in San Francisco, California. The role involves developing and optimizing core knowledge graph components and AI frameworks, focusing on innovation and engineering excellence. The ideal... 
    Flexible hours

    Salesforce

    San Francisco, CA
    2 days ago
  • Role: Lead Data Engineer Location: San Francisco, CA (1-2 days a week onsite is must) Job Type: W2 Contract Contract Length: 9 months with...  ...design patterns, and building scalable BI solutions Knowledge of cloud platforms (Azure, AWS or equivalent) #J-18808-Ljbffr Saransh
    Contract work
    2 days per week
    1 day per week

    Saransh

    San Francisco, CA
    1 day ago
  •  ...a global leader in enterprise-grade data analytics and AI solutions, committed...  ...with EQT. Website: Job Role Lead, Data Science & Data Engineering - Function: Analytics & AI Consulting...  ...designing data pipelines and feature platforms, and translating commercial questions... 
    Worldwide

    Straive

    San Francisco, CA
    1 day ago
  • $200k

     ...company in San Francisco seeks a Developer Experience Lead to enhance internal tools for their pre-training data team. This role involves leading tooling efforts...  .... Candidates should have strong software engineering fundamentals and experience with CI systems. Annual... 

    Magic

    San Francisco, CA
    4 days ago
  • $215.2k - $245.6k

    Capital One National Association is seeking a Lead Data Engineer in San Francisco, CA. The role involves driving technological transformations and collaborating with agile teams to build technical solutions using programming languages like Python, SQL, and Scala. Ideal... 

    Capital One National Association

    San Francisco, CA
    2 days ago
  • $150k - $220k

    Mochi Health, based in San Francisco, is seeking a Data Manager to lead the Data team. The role involves executing analytics and data engineering projects while tracking important KPIs. Candidates should have strong SQL and Python skills and experience managing technical... 

    Mochi-Health

    San Francisco, CA
    4 days ago
  • Tech Lead, Data & Inference Engineer Location: San Francisco Work type: Full Time Compensation: above market base + bonus + equity Roles & Responsibilities...  ...design, development and scaling of an end‑to‑end data platform from ingestion to insights, ensuring that data is fast,... 
    Full time

    Catalyst Labs

    San Francisco, CA
    1 day ago
  •  ...At Sierra, we’re creating a platform to help businesses build better...  ...You’ll join a full-stack data team building the systems making...  ...of these areas: Platform Engineering You have designed, built,...  ...ranking). A track record of leading complex technical projects or... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    21 days ago
  •  ...APIs. We work across research, engineering, product, policy, safety, and...  ...safely. The Trust & Safety Data Engineering team builds the...  ...Science, Engineering, and Data Platform to create reliable, privacy-...  ...Role We are hiring a Technical Lead Manager to lead and grow the... 
    Relocation package

    OpenAI

    San Francisco, CA
    1 day ago
  • $179.5k - $221.5k

     ...understands itself as a business, working closely with Data Science and Analytics. What's unusual here: the platform you're instrumenting is the same one your...  ...partner closely with product analytics, product engineering, and data infrastructure to turn business questions... 
    Remote work
    Shift work

    Airtable

    San Francisco, CA
    4 days ago
  • $130.9k - $220k

     ...Senior Software Engineer - Data Platform Remote– US Samsara’s Mission Improve the safety, efficiency, and sustainability of the operations that power the global economy. About the Role Samsara is looking for a Senior Software Engineer to join our Data Platform team, which... 
    Full time
    Remote work
    Relocation package
    Flexible hours

    Samsara

    San Francisco, CA
    4 days ago
  •  ...Purpose of Onyx The Onyx Research Data Tech organization is GSK’s research data ecosystem that brings...  ...of product and portfolio leadership, data engineering, infrastructure and DevOps, data/metadata/knowledge platforms, and AI/ML and analysis platforms, all geared toward... 
    Work experience placement
    Local area

    GSK LLP

    San Francisco, CA
    5 days ago
  •  ...professionals for this role. About us At Sierra, we’re creating a platform to help businesses build better, more human customer experiences...  ...customers trust Sierra with their most sensitive workflows and data. You’ll help set and uphold a high bar for security, compliance... 
    Full time
    Flexible hours

    Sierra

    San Francisco, CA
    21 days ago
  •  ...Data/ETL Engineer (Founding Team) Location: San Francisco Bay Area Type: Full-Time Compensation: Competitive salary + early-stage equity...  ...problems. About the Role We’re building a multi-tenant, AI-native platform where enterprise data becomes actionable through semantic... 
    Full time

    Fabrion

    San Francisco, CA
    2 days ago
  • $60 - $85 per hour

     ...A fast-growing AI company''s Finance organization is looking for a Data Engineer to develop and maintain the data infrastructure that supports its CFO team. This role offers the opportunity to join a small, dynamic team with a wide-ranging mandate: build data pipelines... 
    Hourly pay
    Full time
    Contract work

    SaidGig

    San Francisco, CA
    5 days ago
  •  ...San Francisco, CA - Hybrid Responsibilities Strong experience in data engineering, data platforms, distributed systems, or enterprise data infrastructure. Practical experience building AI-enabled data systems, retrieval systems, semantic layers, or data agents. Strong... 

    SWITS DIGITAL Private Limited

    San Francisco, CA
    5 days ago
  •  ...Plaid is seeking a Data Engineer in San Francisco to enhance our data systems and build datasets that empower business decisions. This role involves collaboration across teams while advocating for data quality and effective strategies. The ideal candidate should have over... 

    Plaid

    San Francisco, CA
    3 days ago
  •  ...Senior Data Engineer Location: Bay Area (San Francisco Peninsula) or NYC (secondary) The Opportunity...  ..., so you get to design the data platform from scratch instead of untangling years...  ...and sustain them. Independent Leader: You lead complex technical initiatives on your own... 
    Work at office
    Relocation package
    Flexible hours
    2 days per week

    MrBeast

    San Francisco, CA
    8 hours ago
  •  ...staffing services. Quantix ProTech was recently featured in US News and World Report and Forbes. Job Description Job Title: Data Platform Engineer Location: San Francisco, CA Type: Direct Hire Job Description: Our client in the San Francisco, CA area is looking for a... 

    Quantix

    San Francisco, CA
    5 days ago
  •  ...combination of inventive research, design, and engineering. Our organization is very flat, and our...  ...code. About the role Take data foundations from 0-1 on the bleeding edge...  ...data. As one of Cursor’s first Data Platform Engineers, you’ll own the systems that make... 
    Full time
    Immediate start

    Cursor

    San Francisco, CA
    5 days ago
  • $203k - $282k

     ...You will collaborate with Data Scientists, Machine Learning Engineers, and Product Engineers to instrument and quantify our ML tools. Through this role you will have frequent opportunities to build the core data pipelines that enable recommendations to the organizational... 
    Full time
    Work at office
    Remote work
    Home office
    Flexible hours

    Weights & Biases

    San Francisco, CA
    5 days ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer, Data Platform. Be the first to apply!