Lead Data Scientist, Data Products

Define evaluation frameworks, offline and online metrics, and quality bars for internal models and ML products Build the measurement layer for cross-domain machine learning products Scale organizational visibility into performance across the business Own experimentation and monitoring for production models Contribute to monitoring key metrics and drift detection for enriched datasets Ensure quality for downstream athlete-facing experiences Identify and size AI/ML product opportunities Translate ambiguous problem spaces into scoped initiatives with defined success criteria Serve as the data science domain expert for the team Raise standards for baselines, validation, and evidence quality across ML engineers and cross-functional partners Partner closely with Machine Learning Engineers and cross-functional teams Requirements 5+ years of experience in data science or a related quantitative domain Hands-on ownership of model evaluation and measurement for systems running in production Experience defining evaluation frameworks for ambiguous ML problems Experience with baseline selection, offline and online metric design, and validation strategies where ground truth is imperfect Strong SQL proficiency Comfort writing production-quality Python for statistical data processing Fluency in metrics and measurement for consumer software products Ability to translate business needs into technical plans and vice versa Ability to work on-site in the San Francisco office three days per week Must be located within commuting distance of San Francisco or willing to relocate Must address employment visa sponsorship status Core Competencies Demonstrates expertise in defining evaluation frameworks and metrics for machine learning products, with a strong focus on model evaluation, measurement, and data quality. Proficient in translating business needs into technical plans while collaborating effectively with cross-functional teams. Highest-signal resume keywords Model Evaluation SQL Proficiency Python for Data Processing Evaluation Frameworks for ML Metrics and Measurement for Software Hard Skills Model Evaluation Evaluation Frameworks Baseline Selection Metric Design Validation Strategies Statistical Data Processing Machine Learning Data Science Performance Monitoring Drift Detection Soft Skills Collaboration Problem-Solving Communication Industry Keywords Machine Learning Products Consumer Software Data Science Domain Expertise Cross-Functional Teams San Francisco Office #J-18808-Ljbffr Jobtailor