Calibration record

Every number below is computed weekly from live resolutions only, with bootstrap confidence intervals (1000 resamples). When an interval crosses zero we say so; nothing here is a claim until it separates.

snapshot 2026-10-25 05:05 UTC · 291 resolved · 639 scored forecasts · 1047 shadow scores · 89 abstained

Brier by verdict class

the core experiment: does the verdict class carry information about error?

Verdict	n	Brier	Bootstrap CI
CAUTION	200	0.232	95% CI [0.203, 0.263]
CONSTRAINED	439	0.278	95% CI [0.257, 0.300]

ABSTAIN rows never publish a probability, so they cannot appear here; their value is measured below, through the shadow.

Value of abstention

an ungated shadow model forecasts every question; we compare its error on the questions we abstained from

shadow on published set

0.253

n=723

shadow on abstained set

0.266

n=324

abstention value (diff)

+0.013

95% CI [-0.017, 0.044]

Positive means the questions our gate refused were genuinely harder, even for an ungated model. The interval currently crosses zero: direction only, not yet a claim.

Layer vs shadow on the published set

same questions, same evidence pool; the only difference is the decision layer

diff +0.011 (layer minus shadow; negative = layer better) · 95% CI [-0.012, 0.034]crosses zero: not yet separated

Risk-coverage

verdict classes are the natural thresholds of a selective forecaster

Publish up to	Coverage	Risk (Brier)	n
CONSTRAINED	60.3%	0.278	439
CAUTION	87.8%	0.264	639

naive AURC 0.242 · with few points this is descriptive, not comparative