Sign up to access all features of our service.
  • Job search
  • Favorites
  • Create a CV
    New
  • Salaries
  • Subscriptions

Lead Data Engineer, Data Platform

CrewAI

About CrewAI

CrewAI is the leading framework and enterprise platform for building and orchestrating multi-agent AI systems, powering 300M+ agent executions per month across thousands of companies. As the product, platform, and customer base scale, data is becoming one of the most important systems in the company: how we understand usage, reliability, activation, customer health, cost, governance, and where to invest next.

Today, we have meaningful data already, but it is spread across product telemetry, trace data, application databases, analytics tables, Cube models, Metabase dashboards, and team-specific queries. We need someone to turn that into a coherent, trusted, useful data foundation.
The Role

You'll be CrewAI's first dedicated data engineering hire. Your job is to own the data foundation end to end: rationalize what exists, improve the infrastructure, define trusted metrics, close instrumentation gaps, and make data accessible enough that product, growth, engineering, customer success, and leadership can actually use it.

This is a foundational role with real range. The center of gravity is data infrastructure and analytics engineering: pipelines, warehouse/lake design, semantic modeling, metric definitions, data quality, and self-serve access. You'll also be the person who turns messy questions into clear analysis, reliable dashboards, and better product decisions.

This is not a maintenance role. It is a "make data legible and useful for the company" role.
What You'll Do
  • Own and evolve CrewAI's data platform across ingestion, transformation, storage, semantic modeling, BI, and operational data quality.
  • Rationalize the existing data estate: product events, execution telemetry, OpenTelemetry-derived traces, application tables, Cube models, Redshift/data-lake tables, Metabase dashboards, and team-specific reporting.
  • Establish trusted source-of-truth metrics for the business and product, including executions, active builders/users, activation, deployment health, token and cost usage, customer health, governance adoption, retention, and feature usage.
  • Build and maintain the models, pipelines, and metric layers that make those numbers consistent across teams.
  • Partner with product and engineering to improve instrumentation, event taxonomy, data contracts, and telemetry coverage for new features.
  • Make data self-serve through clear dashboards, documented datasets, reusable metric definitions, and sensible access patterns.
  • Improve reliability and trust in the stack through data quality checks, freshness monitoring, lineage, alerting, backfills, and incident/debug workflows.
  • Partner with Discovery, product, and go-to-market teams on analysis behind recommendations, customer signals, usage patterns, and roadmap decisions.
  • Keep the stack secure and cost-aware, including access control, PII handling, retention, and warehouse/query efficiency.
  • Help define how CrewAI uses data internally as the company scales.
Requirements
What We're Looking For
  • Strong data engineering or analytics engineering experience, especially building data foundations in fast-moving product companies.
  • Excellent SQL and data modeling skills, with experience designing reliable datasets, fact/dimension models, and metric definitions.
  • Experience operating a warehouse or analytics store such as Redshift, Snowflake, BigQuery, Postgres, or similar.
  • Familiarity with transformation and modeling tools such as dbt, Cube, semantic layers, or equivalent systems.
  • Experience with event pipelines, product telemetry, application data, and BI tools such as Metabase, Looker, Mode, or similar.
  • Strong Python for data work, automation, validation, and operational workflows.
  • Product sense: you can turn ambiguous questions into useful metrics, and you care whether the numbers are understood correctly.
  • Pragmatism: you are comfortable inheriting messy systems, improving them incrementally, and choosing boring reliable solutions when they are right.
  • Strong communication and documentation habits. You make data easier for other people to use.
  • Comfort being the first dedicated owner in an early-stage, high-growth environment.
Bonus
  • Experience with LLM, agent, observability, trace, usage, or cost analytics.
  • Experience with OpenTelemetry, high-volume event data, or operational telemetry.
  • Experience with experimentation, causal analysis, activation/retention modeling, or customer health scoring.
  • Experience defining event taxonomies and instrumentation standards for SaaS products.
  • Familiarity with Rails/Postgres application data, background jobs, and product analytics in B2B SaaS.
  • Lightweight ML or recommendation experience, especially where it supports product or customer workflows.
Vacancy posted 1 day ago
Similar jobs that could be interesting for youBased on the Lead Data Engineer, Data Platform in San Francisco, CA vacancy
  • Ektello is seeking a Lead Data Engineer to join the Supply Chain Data & AI team in Pleasanton, CA. You will lead the design, development, and...  ...leadership and deep expertise in building cloud-native data platforms on Google Cloud Platform (GCP). You will collaborate with... 
    Suggested

    ektello

    San Francisco, CA
    3 days ago
  • We are seeking a hands-on Lead Data Engineer to join the Data & AI Supply Chain organization and drive the design, development, and delivery of enterprise-scale data products on Google Cloud Platform (GCP). This role will support Supply Chain initiatives across Sourcing... 
    Suggested
    Hourly pay
    Freelance

    Solomon Page

    San Francisco, CA
    2 days ago
  • Saransh Inc is hiring a Lead Data Engineer in San Francisco, CA. The role involves building data ingestion pipelines and designing data warehouse architecture. Candidates should have a minimum of 8-10 years in software development with strong experience in SQL, Databricks... 
    Suggested
    Contract work
    2 days per week
    1 day per week

    Saransh

    San Francisco, CA
    22 hours ago
  •  ...employees and our community. Join us in using data, bold thinking, and a commitment to...  ...reporting, and collections. As a Lead Data Engineer, you'll lead the Financial Data Operations...  ...solutionsDrive adoption of modern data platform capabilities, including Databricks, dbt... 
    Suggested
    Full time
    Work at office
    Local area
    Remote work
    Relocation
    Flexible hours

    Lending Club

    San Francisco, CA
    2 days ago
  • $160k - $220k

     ...Lead Data Engineer Location: San Francisco, CA, United States Location Type: On-site Salary Range: 160000 - 220000 USD Annually We are...  ..., and lead the development of scalable, cloud-based data platforms that support enterprise analytics, operational reporting, and... 
    Suggested

    Q-Cells

    San Francisco, CA
    1 day ago
  •  ...Lead Data Engineer The Office of Information Technology (IT) is responsible for enabling State Bar's internal and external stakeholders...  ...technologies. As the technical authority for the Bar's data platform, you will lead the development of scalable, automated data pipelines... 
    Work at office

    GovernmentJobs.com

    San Francisco, CA
    2 days ago
  •  ...Lead Data Engineer The Office of Information Technology (IT) is responsible for enabling State Bar's internal and external stakeholders...  ...technologies. As the technical authority for the Bar's data platform, you will lead the development of scalable, automated data pipelines... 
    Work at office

    State Bar CA

    San Francisco, CA
    5 days ago
  •  ...work on, we'd like to talk. Your Team The data team at Nuna is an interdisciplinary group...  ...actuarial science, and research. Within it, data engineering is the technical backbone which creates the architecture, platform, and operations that everything else is built... 

    Nuna Inc

    San Francisco, CA
    3 days ago
  •  ...Job Title Mandatory Skills: (Oracle or PostgreSQL) and ETL Pipelines and Big Data and AWS Responsibilities • Uses structured tools for analysis and presentation of concepts and models to enhance the BRD • Develop, maintain and deliver training materials to the... 
    Work experience placement

    Omega Solutions Inc

    San Francisco, CA
    3 days ago
  •  ...Lead Data Engineer With MarTech Location: SFO, CA (Hybrid 2 days a week) Key Responsibilities Lead end-to-end MarTech engineering...  ...Skills & Experience Deep expertise in MarTech platforms with hands-on experience in Adobe Campaign or similar orchestration... 
    Remote work
    2 days per week

    Georgia IT Inc

    San Francisco, CA
    3 days ago
  • Everforth Apex Systems is seeking a Lead Data Engineer in San Francisco to design and deliver cloud-native data platforms on Google Cloud. You will lead data pipeline development, data modeling, and analytics solutions across Sourcing, Transportation, and WMS domains. The... 

    Apex Systems

    San Francisco, CA
    4 days ago
  • Expedite Talent Solutions seeks a highly technical Lead Consultant for Data Management, MDM, and Stibo STEP PIM integration. This on-site/hybrid role spans master data governance, ERP and e-commerce data flows, and inventory lifecycle management. You will build and optimize... 

    Expedite Talent Solutions

    San Francisco, CA
    5 days ago
  • $112.5k - $149.97k

    Annual Salary Range Lead IT Analyst: $112,497 - $149,972 FLSA Exempt / Union Represented...  ...Technology is seeking a Lead Data Engineer to join our dynamic team at the State Bar...  ...technical authority for the Bar’s data platform, you will lead the development of scalable... 
    Work at office
    Remote work

    State Bar of California

    San Francisco, CA
    5 days ago
  • Salesforce, Inc. is seeking a Data Engineer for its Enterprise Knowledge Graph & AI team in San Francisco, California. The role involves developing and optimizing core knowledge graph components and AI frameworks, focusing on innovation and engineering excellence. The ideal... 
    Flexible hours

    Salesforce

    San Francisco, CA
    2 days ago
  • Stepping Up Solutions in San Francisco is looking for a talented Lead Data Engineer to join their Data Technology team. In this role, you will design and implement scalable data infrastructure and processes while working closely with product management and data analysts... 

    Stepping Up Solutions

    San Francisco, CA
    4 days ago
  •  ...build intelligent software that automates media buying, making big data actionable for marketers and connects them to their next customer. Based in San Francisco, we are seeking a talented Lead Data Engineer. RadiumOne generates first‑party data about actual customers —... 

    Stepping Up Solutions

    San Francisco, CA
    4 days ago
  • Role: Lead Data Engineer Location: San Francisco, CA (1-2 days a week onsite is must) Job Type: W2 Contract Contract Length: 9 months with...  ...design patterns, and building scalable BI solutions Knowledge of cloud platforms (Azure, AWS or equivalent) #J-18808-Ljbffr Saransh
    Contract work
    2 days per week
    1 day per week

    Saransh

    San Francisco, CA
    22 hours ago
  • $149.3k - $200.2k

     ...come. Reach & Scale: The products and platforms this group builds and operates delight...  ...sports, entertainment & news. The Product & Data Engineering team is responsible for end to end...  ..., messaging and data. Job Summary As a Lead Data Engineer in the Data Platforms team... 
    Work experience placement
    Worldwide

    The Walt Disney Company

    San Francisco, CA
    1 day ago
  • $215.2k - $245.6k

    Lead Data Engineer Do you love building and pioneering in the technology space? Do you enjoy solving complex business problems in a fast-paced, collaborative, inclusive, and iterative delivery environment? At Capital One, you'll be part of a big group of makers, breakers... 
    Full time
    Part time
    Internship
    H1b
    Local area

    SwiftCruit

    San Francisco, CA
    1 day ago
  • State Bar of California is seeking a Lead Data Engineer to modernize its data infrastructure using Microsoft cloud technologies. You will lead scalable data pipelines, govern data, and drive cross-functional data solutions to support analytics, AI, compliance, and operational... 
    Remote job

    State Bar CA

    San Francisco, CA
    5 days ago
  • $160k - $210k

     ...Our fleet produces a continuous stream of data, and we're looking for a senior IC to...  ...insight: for customers, partners, operations, engineering, and leadership. You'll be the bridge...  ...metrics, partnering with the Platform team that owns ingestion. Define the... 
    Full time
    Temporary work
    Flexible hours

    Agtonomy

    South San Francisco, CA
    more than 2 months ago
  • $289.7k - $362.1k

     ...individual tool was revolutionary, but because the platform made everyone else more productive.We're looking for a Head of Data Engineering & Platform who can build the data platform...  ..., and work with.In this role, you'll lead and grow a high-performing team of data engineers... 

    Mercury

    San Francisco, CA
    2 days ago
  •  ...About Omni Omni is the AI analytics platform that turns company data into a trusted source of truth for AI. Built on a semantic model, Omni gives...  ...experience building data platforms, backend systems, workflow engines, or data / ML infrastructure ~ Deep experience with... 
    Full time

    Omni

    San Francisco, CA
    9 hours ago
  • $196k - $261k

     ...our users with the tools they can build in Notion. The Data Product Platform team is responsible for designing, implementing, scaling and...  ...powered product experiences. You’ll join a team of talented engineers in our mission to build the best in class foundational... 
    Full time
    Temporary work
    Local area

    Notion

    San Francisco, CA
    9 hours ago
  • $227.33k - $312.58k

    We’re looking for a Staff ML Data Engineer to join Procore’s AI & Frontier Models organization...  ...opening!What you’ll doAct as the technical lead for data engineering efforts supporting...  ...tracking, reproducibility toolingData Platforms: Databricks, Spark, lakehouse architectures... 
    Full time
    Work at office
    Local area
    Immediate start
    3 days per week

    Procore Technologies

    San Francisco, CA
    2 days ago
  • $300k - $330k

     ...only unified payments and financial platform for global businesses. Powered by...  ...US$11 billion and backed by world-leading investors including T. Rowe Price,...  ...Role Summary:We’re hiring a Director, Data and Knowledge Platform Engineering (based in San Francisco) to own the... 
    Temporary work
    Local area
    Worldwide

    Airwallex

    San Francisco, CA
    1 day ago
  • $172.5k - $260.1k

     ...level-up your career at the company leading workforce transformation in the agentic...  ...Member of Technical Staff, Release Engineering, Agentic Platform (Java / Node.js / iOS)We're hiring an...  ...information about how we use your personal data and your rights, including with... 
    Full time

    Salesforce

    San Francisco, CA
    9 hours ago
  • $225k - $290k

     ...(NYSE: CRCL) is one of the world’s leading internet financial platform companies, building the foundation of...  ...will define and drive the strategy for data reliability, quality, and...  ...investments, define best-in-class data engineering practices, and lead complex, high-impact... 
    Flexible hours

    Circle

    San Francisco, CA
    2 days ago
  •  ...addresses.About the roleRippling's Object Platform is the foundation upon which every...  ...core frameworks that define how business data is modeled, connected, queried, and...  ...the company.We're looking for a Senior Engineering Manager to lead a team within this organization and... 
    Work at office
    3 days per week

    Rippling

    San Francisco, CA
    3 days ago
  •  ...requiredDefense experience and experience working with data at various classification levels...  ...of professional experience as a data engineer, with a strong focus on cloud- based data...  ...(Azure, GCP) a plusPrevious enterprise platform experience preferredSignificant previous... 
    Apprenticeship
    Easy work

    McKinsey & Company

    San Francisco, CA
    1 day ago

Do you want to receive more vacancies?

Subscribe and receive similar vacancies to Lead Data Engineer, Data Platform. Be the first to apply!