Bayesian A/B Testing Expert

Design, analyze, and interpret A/B tests using Bayesian statistical methods. Provides posterior distributions, credible intervals, expected loss calculations, and actionable recommendations for experiment-driven decision making.

Gold

v1.0.00 activationsData & AnalyticsTechnologyexpert

SupaScore

84.35

Research Quality (15%)

8.5

Prompt Engineering (25%)

8.5

Practical Utility (15%)

8.3

Completeness (10%)

8.4

User Satisfaction (20%)

8.3

Decision Usefulness (15%)

8.6

Best for

▸Design Bayesian A/B tests for SaaS conversion rate optimization with proper prior selection
▸Analyze ongoing experiments using posterior distributions and credible intervals
▸Calculate expected loss for making decisions before statistical significance
▸Implement Thompson Sampling for multi-armed bandit optimization
▸Design sequential testing frameworks that avoid peeking problems

What you'll get

●Complete experiment design with Beta(α,β) prior specifications, sample size calculations using posterior precision method, and stopping criteria based on credible interval width
●Posterior analysis showing probability distributions for each variant, 95% credible intervals, P(B > A) calculations, and expected loss tables for different decision thresholds
●Thompson Sampling implementation with dynamic allocation recommendations, regret bounds analysis, and performance monitoring dashboards

Not designed for ↓

×Classical frequentist hypothesis testing with p-values and power analysis
×Machine learning model A/B testing without proper causal inference setup
×Marketing attribution modeling or multi-touch attribution analysis
×General statistical consulting outside of controlled experimentation

Expects

Clear hypothesis, primary metric definition, historical baseline data, business context for minimum detectable effect, and decision framework for experiment outcomes.

Returns

Complete Bayesian analysis including posterior distributions, credible intervals, expected loss calculations, probability statements, and actionable business recommendations with statistical justification.

Evidence Policy

Enabled: this skill cites sources and distinguishes evidence from opinion.

bayesian-statisticsab-testingexperimentationposterior-distributioncredible-intervalsexpected-lossthompson-samplingconversion-optimizationhypothesis-testingprior-elicitationbeta-binomialdecision-theory

Research Foundation: 8 sources (3 books, 2 paper, 3 official docs)

This skill was developed through independent research and synthesis. SupaSkills is not affiliated with or endorsed by any cited author or organisation.

Version History

v1.0.02/16/2026

Initial release

Prerequisites

Use these skills first for best results.

Bayesian Statistics ExpertPlatinum

Works well with

A/B Test AnalystPlatinum Conversion Rate OptimizerGold Experimental Design SpecialistGold Hypothesis Testing ExpertGold Statistical Analysis AdvisorPlatinum

Need more depth?

Specialist skills that go deeper in areas this skill touches.

Causal Inference AnalystGold Monte Carlo Simulation ExpertGold

Common Workflows

Complete Bayesian Experimentation Pipeline

Design Bayesian experiment, analyze results during runtime, then optimize based on findings for continuous improvement

bayesian-ab-testing-expert→A/B Test Analyst→Conversion Rate Optimizer

Activate this skill in Claude Code

Start Free to Activate This Skill