Experimentation Analytics Lead

Kforce has a client in San Francisco, CA that is looking for a Lead Experimentation Analyst to drive rigorous experimentation and measurement across our digital product portfolio. This role will serve as a practitioner and subject matter expert for website experimentation, partnering with Product, UX, Analytics, and Engineering teams to design high-quality experiments, evaluate results, and ensure product decisions are grounded in statistically sound evidence. Responsibilities:

• Partner with Product and UX teams to develop hypotheses, success metrics, and experimentation strategies for digital experiences
• Design A/B and multivariate experiments, including audience allocation, sample sizing, test duration, primary KPIs, and guardrail metrics
• Conduct statistical power analysis and determine minimum detectable effects for proposed experiments
• Analyze experiment results and determine statistical and practical significance
• Apply advanced experimentation techniques including variance reduction, sequential testing, heterogeneous treatment effect analysis, and other appropriate statistical methodologies
• Evaluate experiment validity, including sample ratio mismatch, instrumentation issues, novelty effects, traffic imbalance, and other sources of bias
• Develop measurement approaches for tests with uneven audience allocation or operational constraints
• Conduct segmentation and follow-up analyses to understand how experiment effects differ across devices, customer groups, product categories, and experiences
• Establish experimentation standards, governance, and best practices across digital product teams
• Partner with analytics implementation teams to validate tracking and ensure experiment data quality
• 5+ years of experience in experimentation, product analytics, statistics, data science, or a related analytics field
• Strong statistical foundation including hypothesis testing, power analysis, confidence intervals, variance, and experimental design
• Demonstrated experience designing and analyzing digital A/B tests and multivariate experiments
• Strong SQL skills and experience working with large-scale behavioral or event-level datasets
• Experience with experimentation platforms such as Adobe Target, Optimizely, Statsig, LaunchDarkly, or similar tools
• Familiarity with advanced experimentation approaches such as CUPED, sequential testing, Bayesian methods, causal inference, or heterogeneous treatment effects
• Strong understanding of ecommerce product metrics and digital customer journeys
• Ability to identify experiment quality issues and explain statistical concepts clearly to non-technical stakeholders
• Experience with Python or R preferred
• Excellent communication skills and ability to turn experiment results into clear business recommendations