Model Performance & Detection System Ops Intelligence

CardShield Inc. • Champion Model v4.2 • 38M monthly transactions • Last updated: 2 min ago

Model Attributable Loss Prevention
$2.34M
vs $2.1M target
PSI (7D vs Training)
0.18
+0.04 vs baseline
Feature Importance Stability
0.72
rank shifts detected
Score Calibration Drift
0.03
-0.01 vs last week
Champion-Challenger AUC Δ
+0.024
v4.3 ahead
Prediction Latency P99
47ms
-8ms vs target
Feature Store Freshness
99.4%
stable

Production Score Distribution Stability

Training baseline vs Current 7D • PSI: 0.18 • Key driver: Mobile payment proportion shift 41% → 58%

SHAP Feature Importance Rank Evolution

Top 10 features • Training vs Current 7D • txn_velocity_15min: rank 2 → 9

Score Calibration by Risk Band

Predicted vs Observed fraud rate • Drift: 0.03

Champion vs Challenger AUC by Typology

v4.2 vs v4.3 (shadow mode)

False Negative Rate by Score Band

High-risk band (≥0.8) FNR: 2.1%

Fraud Typology Coverage Rate

Detection rate by fraud category

Model Version Performance Registry

Version Status Deployed AUC Precision@0.5 PSI (7D) Loss Prevention Features Latency P99
v4.3 Shadow — 0.946 0.883 0.14 — 127 51ms
v4.2 Champion 61d ago 0.922 0.871 0.18 $2.34M 118 47ms
v4.1 Retired 143d ago 0.908 0.854 0.31 $2.08M 112 43ms
v4.0 Retired 201d ago 0.895 0.839 0.42 $1.91M 104 39ms

Alert Volume Stability Control Chart

Daily alert volume • UCL: 4,820 • LCL: 2,140 • Mean: 3,480

Retraining Trigger Lead Time

Days until intervention threshold
PSI Threshold (0.25) 18 days
AUC Degradation (-0.02) 34 days
FNR Breach (>3%) 28 days
Calibration Drift (>0.08) 41 days
Earliest trigger: PSI in 18 days

Feature Store Freshness by Group

% features updated within SLA window (15 min)

Prediction Latency Breakdown

P99 latency by pipeline stage

Model Health Timeline with Event Annotations

Key events and metric trends over deployment lifecycle
Model Deployment
API Change
Population Shift
Retraining