Design and analyze A/B tests for digital products.
A/B Test Analyst
Experiment Design, Statistical Analysis, CUPED
Best for
- ▸Calculate sample sizes for subscription conversion rate experiments with proper power analysis
- ▸Analyze A/B test results with CUPED variance reduction to detect smaller effect sizes
- ▸Design multi-variant experiments with Bonferroni correction for marketing landing pages
- ▸Implement sequential testing with alpha spending functions for early experiment termination decisions
What you'll get
- ▸Statistical analysis report with t-test results, confidence intervals, effect size interpretation, and business impact recommendations
- ▸Experiment design document with randomization strategy, sample size justification, statistical framework choice, and success criteria
- ▸Post-experiment analysis with CUPED-adjusted metrics, sequential testing boundaries, and guardrail metric validation
Clear experiment hypothesis with defined primary metric, baseline conversion rates, minimum detectable effect size, and business context about the change being tested.
Rigorous experimental design with sample size calculations, statistical framework selection, analysis plan with proper significance testing, and actionable business recommendations with confidence intervals.
What's inside
“You are a Senior A/B Test Analyst. You design rigorous experiments, analyze results with statistical integrity, and translate findings into actionable business decisions. - **Prevent false positives** through pre-registered hypotheses, sequential testing corrections, and Sample Ratio Mismatch detect...”
Covers
Not designed for ↓
- ×General data analysis or business intelligence dashboard creation
- ×Machine learning model training or predictive analytics
- ×Survey design or market research methodology
- ×Database query optimization or data pipeline architecture
SupaScore
88.88▼
Evidence Policy
Standard: no explicit evidence policy.
Research Foundation: 8 sources (3 books, 2 official docs, 3 paper)
This skill was developed through independent research and synthesis. SupaSkills is not affiliated with or endorsed by any cited author or organisation.
Version History
v5.5 final distill
Pipeline v4: rebuilt with 3 helper skills
Auto-versioned: masterfile quality gate passed (score: 86.0)
Initial release
Works well with
Need more depth?
Specialist skills that go deeper in areas this skill touches.
Common Workflows
Growth Optimization Pipeline
Design growth hypotheses, run statistically rigorous experiments, then optimize based on validated learnings
© 2026 Kill The Dragon GmbH. This skill and its system prompt are protected by copyright. Unauthorised redistribution is prohibited. Terms of Service · Legal Notice