claimkeeper

A research note makes claims. Most are never checked.

claimkeeper is a ledger of the falsifiable ones. Each is a sentence plus a test: a metric over a data file, an operator, a threshold, the date it can be judged, and the probability the author attached to it. Once the date arrives and the data exist, a resolver scores the claim from the data and writes the verdict, the observed number and the threshold into the ledger.

A scoreboard gives hit rate and Brier score per report. A bot answers /claims, /due, /claim, /resolve and /score, and a watcher posts the digest when something resolves.

The one rule

A claim is judged by the number, never by the author. The resolver runs the test the report registered, reports the observed value beside the threshold, and does not round in anyone’s favour. A card reads:

demo.cpi_through [WRONG] — made 2026-02-06, resolves 2026-02-11. Through the 11 Feb CPI the put wing cheapens. test: table.change < 0, stated p = 0.70. observed 0.202989 → WRONG.

What’s tested

The six operators exactly. Every resolver against a known table. The ledger’s one-way status moves and its append-only log. The Brier arithmetic. The bot’s commands over a scripted transport. Thirty-four tests, and the demo scoreboard is regenerated and diffed in CI.