A/B Testing Dashboard

Revenue Impact & Monetization Attribution · NovaSpark Commerce

Q3 2024 Performance
$4.2M
Annualized Revenue Impact
RPV Lift %
+12.4%
+2.1% vs Q2
CUPED Adj. Lift
+9.8%
+1.4% vs Q2
SRM Rate
8.3%
Above 5% threshold
Avg Decision Time
11.2d
-2.8d vs Q2
Portfolio ROI
3.8x
+0.6x vs Q2
Experiments Shipped
18
14 revenue-positive

Incremental Revenue Waterfall

Q3 2024 annualized impact by experiment stage

RPV by Variant — Top 8 Experiments

Novelty Effect Decay Index

AOV Variance by Variant

Refund & Return Rate Differential

Experiment Performance Table

Experiment Variant RPV Lift % CUPED Adj % AOV Ann. Revenue SRM Days
EXP-Q3-014 Checkout: 1-Click +18.2% +15.1% $142 $820K — 9d
EXP-Q3-007 Pricing: Tiered +14.6% +12.3% $156 $680K — 12d
EXP-Q3-011 Homepage: Hero CTA +11.8% +9.2% $138 $540K 14d
EXP-Q3-018 Onboarding: Steps +9.4% +7.8% $145 $480K — 11d
EXP-Q3-003 Retention: Email +7.2% +6.1% $133 $390K — 10d
EXP-Q3-009 Cart: Promo Banner -2.1% -1.8% $128 -$120K 8d
EXP-Q3-015 PDP: Reviews Widget +5.3% +4.2% $141 $310K — 13d
EXP-Q3-012 Checkout: Trust Badge +3.8% +3.1% $136 $180K 15d

LTV-Adjusted Cohort Delta

Opportunity Cost by Decision Time

Experiment Performance Heatmap — RPV Lift % by Surface × Week

W1
W2
W3
W4
W5
W6
W7
W8
W9
W10
W11
W12
Checkout
+18.2
+17.8
+16.9
+15.1
+14.2
+13.8
+13.1
+12.4
+11.9
+11.3
+10.8
+10.2
Pricing
+14.6
+14.1
+13.4
+12.8
+12.3
+11.7
+11.2
+10.6
+10.1
+9.5
+9.0
+8.4
Homepage
+11.8
+11.2
+10.6
+10.1
+9.5
+9.2
+8.8
+8.3
+7.9
+7.4
+7.0
+6.5
Onboarding
+9.4
+9.0
+8.6
+8.2
+7.8
+7.5
+7.1
+6.8
+6.4
+6.1
+5.7
+5.4
Retention
+7.2
+6.9
+6.6
+6.3
+6.1
+5.8
+5.6
+5.3
+5.1
+4.8
+4.6
+4.4
PDP
+5.3
+5.1
+4.9
+4.7
+4.5
+4.3
+4.2
+4.0
+3.8
+3.7
+3.5
+3.4
Active Experiments
45
18 shipped in Q3
Win Rate
77.8%
14 of 18 revenue-positive
Target: Annualized Revenue
$2.0M
210% of target achieved