Staff, Backend Engineer - Catalog

Staff, Backend Engineer - CatalogPalo Alto, California, United StatesDataHub is an AI & Data Context Platform adopted by over 3,000 enterprises, including Apple, CVS Health, Netflix, and Visa. Innovated jointly with a thriving open-source community of 13,000+ members, DataHub's metadata graph provides in-depth context of AI and data assets with best-in-class scalability and extensibility.The company's enterprise SaaS offering, DataHub Cloud, delivers a fully managed solution with AI-powered discovery, observability, and governance capabilities. Organizations rely on DataHub solutions to accelerate time-to-value from their data investments, ensure AI system reliability, and implement unified governance, enabling AI & data to work together and bring order to data chaos.The ChallengeAs AI and data products become business-critical, enterprises face a metadata crisis:No unified way to track the complex data supply chain feeding AI systemsEngineering teams struggling with data discovery, lineage, and governanceOrganizations needing machine-scale metadata management, not just human-browsable catalogsWhy This MattersThis is where infrastructure meets impact. The metadata layer you'll build will directly power the next generation of AI systems at massive scale. Your code will determine how safely and effectively thousands of organizations deploy AI, affecting millions of users worldwide.The RoleWe're looking for an exceptional Staff, Backend engineer to lead development of DataHub's Platform framework – the core that connects diverse data systems and powers our metadata collection capabilities.You'll BuildScalable, fault-tolerant ingestion systems for enterprise-scale metadataClean, intuitive APIs for our connector ecosystemEvent-driven architectures for real-time metadata processingSchema mapping between diverse systems and DataHub's unified modelVersioning systems for AI assets (training data, model weights, embeddings)You Have8+ years building production-grade distributed systemsAdvanced Python and API design expertiseExperience with high-scale data processing or integration frameworksStrong systems knowledge and distributed architecture experienceProven track record solving complex technical challengesBuilt and maintained online applications serving live traffic at scale (100+ QPS)Set up monitoring and alerting for servicesDesigned indexing, storage, and data architectures to make large-scale data accessible to online servicesDesigned and scaled distributed systemsHands-on experience developing in a tight loop with LLMs and applying best practices for scalable LLM developmentLanguagesOne of Java/Scala/Kotlin/C#/Go - very strong nice-to-have / borderline must-havePython/TypeScript/Node.js - nice-to-haveTechnical SkillsAWSKubernetes/DockerCI/CD deployment pipelinesMicroservice ArchitectureBonus PointsExperience with DataHub or similar metadata/ETL frameworks (Airflow, Airbyte, dbt)Open-source contributionsExperience building and maintaining services that make calls to LLMs in order to serve live trafficExperience fine-tuning LLM-powered applications exposed to end usersEarly-stage startup experienceLocation and CompensationBay Area (hybrid, 3 days in Palo Alto office)Salary Range: $225,000 to $300,000Benefits and PerksWe invest in people so they can do their best work and enjoy doing it. Our benefits reflect the way we build: practical, thoughtful, and designed to support long-term growth.Competitive compensationWe offer salaries that reflect your skills, experience, and the impact you make. You bring value—we make sure you're recognized for it.Equity for everyoneEvery team member receives an ownership stake in the company. When we grow, you grow with us.Remote WorkAll roles are remote unless otherwise specified in the job description. Review the job description to confirm if the role you are interested in is remote or hybrid.Location flexibilityHome office, coworking space, or something in between? We support your ideal setup. You'll receive a monthly coworking stipend to use whenever you need a change of pace or in-person collaboration time.Comprehensive health coverageYour well-being matters. We cover 99% of medical, dental, and vision premiums employees, and 65% for dependents.Flexible savings accountsWe offer FSAs to help cover planned or unexpected healthcare costs. You can also opt into a Dependent Care FSA to support family needs.Support for every path to parenthoodThrough Carrot Fertility, we provide inclusive fertility benefits and family-forming support. All U.S. employees have access, regardless of age, gender identity, or family structure.Time off that works for youWe trust you to take the time you need. Our unlimited PTO and sick leave policy is designed for flexibility, rest, and real life.Why Join UsDataHub is at a rare inflection point: we've achieved product-market fit, earned the trust of leading enterprises, and secured backing from top-tier investors like Bessemer Venture Partners and 8VC. The context platform market is expected to grow from $1B to $9B in the next five years—and we're leading the way.By joining our team, you'll:Tackle high-impact challenges at the heart of enterprise AI infrastructureShip production systems that power real-world use cases at global scaleCollaborate with a high-caliber team of builders who've scaled some of the most influential data tools in the worldBuild the next generation of AI-native data systems, including conversational agents, intelligent classification, automated governance, and moreIf you're passionate about technology, enjoy working with customers, and want to be part of a fast-growing company changing the industry, we want to hear from you!