A/B Testing Dashboard

Experiment Velocity & Statistical Rigor — Meridian Commerce

Throughput Rate
4.8/mo
+12%
Statistical Validity
91.2%
+3.2%
Win Rate (Shipped)
22.4%
0%
Median Runtime
19d
-14%
False Discovery
4.7%
-1.1%
Incremental Lift
$2.8M
+18%

Experiment Throughput & Median Runtime

Win Rate by Experiment Category

Decision Latency Distribution

MDE vs. Observed Effect Size

SRM Detection Rate Over Time

Experiment Collision Index by Surface & Month

Jan
Feb
Mar
Apr
May
Jun
Checkout
0.12
0.28
0.15
0.11
0.09
0.13
Onboarding
0.08
0.10
0.06
0.14
0.22
0.11
Email
0.05
0.07
0.04
0.08
0.06
0.09
Pricing
0.19
0.34
0.25
0.16
0.12
0.14
Navigation
0.11
0.15
0.09
0.21
0.17
0.13

Index: avg. daily overlap %. Green <0.15, Yellow 0.15–0.25, Red >0.25. Feb Pricing spike: concurrent price-test + discount logic experiments.

Cumulative Incremental Lift (Shipped Winners)

False Discovery Rate Estimate

Experiment Pipeline — Recent Conclusions

Experiment ID Surface Hypothesis Start Date Runtime Sample Size Result Lift Decision Status
EXP-2406-091 Checkout Single-page checkout flow May 12 18d 124,400 Significant +3.2% Shipped $420K ARR
EXP-2406-088 Pricing Annual discount 15% → 20% May 8 21d 89,200 Significant +5.8% Shipped $780K ARR
EXP-2406-085 Email Weekly digest time shift Apr 29 14d 212,000 No effect +0.4% Archived —
EXP-2405-079 Onboarding Video tutorial on step 2 Apr 18 19d 54,300 Significant +2.1% Shipped $290K ARR
EXP-2405-074 Navigation Sticky header on scroll Apr 9 16d 178,900 No effect -0.2% Archived —
EXP-2405-071 Checkout Trust badge above CTA Apr 2 22d 105,600 Significant +1.9% Shipped $340K ARR
EXP-2404-066 Pricing Feature comparison table Mar 22 25d 92,100 SRM Detected — Invalidated —
EXP-2404-062 Email Subject line personalization Mar 15 17d 198,400 Significant +4.1% Shipped $520K ARR