Track record

Every forecast Kestrel publishes is a dated commitment, frozen when it is made. This page is what happened next: every file that has been decided, the range we published before it was, and whether the decision landed inside that range.

Why there is no accuracy percentage here yet

The record, by permit type

Withdrawn forecasts

A forecast is withdrawn only when it was produced with a defect we can demonstrate without reference to how it scored, and never because it scored badly. Nothing is deleted: the withdrawal is recorded, dated and published here, and the withdrawn forecast is out of the record above.

How this record is built

Every file is graded against the first range we published for it. When a file enters coverage we publish a range, and that is the range it is scored against, however many times we forecast it afterwards. The alternative, scoring the last forecast made before the decision landed, sounds equivalent and is not: a file about to be decided is always being forecast from a few days out, so a last-forecast record measures how close to the end we were standing rather than how good the forecast was.

Every exclusion is stated. A file we could not grade is listed with the reason we could not grade it, and the reason was written down before it was applied. The commonest is a decision the register published weeks after it was made, so the file was still showing as pending when we forecast it: no forward-looking range can contain a decision already made.

An annulled question counts neither way. The record can strike a question that should never have been asked. It is struck with a published reason, and it is not quietly deleted.

The ranges are 80% credible intervals. That is a claim about how often reality should land inside them: about eight times in ten. The counts on this page are the evidence for or against that claim, which is why they are published at the same size as the forecasts themselves.

Only live forecasts count. Retrospective tests against history are never pooled with this record and never appear here.