Calibration

per-model accuracy on resolved markets - Brier, log-loss, reliability curve

Per-model scores

Brier: lower better, 0.25 = naive 50/50. Log-loss: lower better. Hit rate: fraction where the model picked the winning side.

Replay every resolved decision in the window through the live sizing/side logic. Per-tier hypothetical PnL.

Lookback (days) Min conviction

Predicted-prob bucket vs actual frequency. Perfectly-calibrated model: mean_actual ~ mean_predicted in every bucket.

Role Model Buckets

Bucket	N	Mean predicted	Mean actual	Calibration