Rule Governance & Model Performance Dashboard

NorthShore Payments • Production Model v3.2.1 • 14 Weeks Since Deploy

Portfolio Gini

0.741

-3.5% vs. deploy

KS Statistic

0.512

-2.1%

PSI (Score Drift)

0.180

Threshold breach

Rule Redundancy Index

38.2%

-4.3% MoM

iFCR (Rules Over Model)

12.4%

+1.8%

Calibration Error (ECE)

0.034

+0.009

Threshold Sensitivity Curve

Model v3.2.1 • Current threshold: 0.45

Score Distribution Stability (PSI)

Rule Library Performance Scatter

Bubble size = Volume • Color = Redundancy vs. Model

Champion vs. Challenger Score Divergence

Calibration Plot (Expected vs. Observed Fraud)

Model Performance History

Week	Gini	KS	PSI	ECE	Approval %	Fraud Rate	Status
Week 1	0.768	0.523	0.042	0.025	87.2%	0.84%	Healthy
Week 4	0.761	0.519	0.068	0.027	86.8%	0.87%	Healthy
Week 8	0.752	0.515	0.112	0.029	86.3%	0.91%	Watch
Week 10	0.747	0.514	0.145	0.031	85.9%	0.94%	Watch
Week 12	0.743	0.513	0.168	0.033	85.4%	0.97%	Drift Alert
Week 14 (Current)	0.741	0.512	0.180	0.034	85.1%	0.99%	Action Required

Rule Interaction Conflict Rate

Rules with overlapping coverage (>60% shared transactions)

R-047 (Velocity Check) ↔ R-082 (High Frequency)

84.3% transaction overlap • Redundant decline logic

84.3%

R-015 (Amount Threshold) ↔ R-091 (Micro-Txn Pattern)

71.2% overlap • Consider merging into single rule

71.2%

R-023 (New Account Guard) ↔ Model Score Decile 1-2

67.8% overlap • Model already flags new accounts

67.8%

R-102 (Device Fingerprint) ↔ R-134 (IP Reputation)

52.1% overlap • Acceptable complementary coverage

52.1%

Top Rules by Incremental Value

R-134: IP Reputation

+8.2% fraud capture over model

8.2%

R-091: Micro-Txn Pattern

+6.4% iFCR

6.4%

R-102: Device Fingerprint

+5.7% iFCR

5.7%

R-047: Velocity Check

+2.1% iFCR (Redundant)

2.1%

R-023: New Account Guard

+0.8% iFCR (Retire candidate)

0.8%

Governance Action Queue

Priority	Action	Rationale	Impact	Owner	Status
P0	Promote Challenger v3.3.0 to Production	PSI breach 0.180 + Gini decline 3.5%. Challenger shows +4.2% Gini in shadow mode.	+1.8% approval	Model Risk Committee	Pending Approval
P1	Retire Rules R-047, R-082 (Redundant Velocity)	84.3% overlap. Only 2.1% incremental fraud capture. High false positive rate.	+0.9% approval	Rule Governance Lead	Approved
P1	Recalibrate R-023 Threshold (New Account Guard)	Model now flags new accounts effectively. Reduce rule threshold from 0.3 to 0.5 to reduce FPR.	+0.6% approval	Analytics Team	In Progress
P2	Add Feature: Merchant Category Code (MCC) for Gaming/Streaming	PSI shows micro-transaction drift. Training set underrepresented MCC 5816, 7995.	+0.8% Gini	Feature Engineering	Backlog