Trained Calibration

Sources#

Summary#

Calibration — expressing the right amount of confidence, including on unsettled questions — treated as a training target with its own reward design, rather than a byproduct of scale or a constitution line. The concrete recipe is TML's "epistemics" stack for Inkling (July 2026, vendor-claim), the first release in this wiki to describe end-to-end how calibration gets trained in:

RL against proper scoring rules on a large corpus of resolved real-world questions — the forecasting-theory device (a scoring rule is proper when truthful probability reporting maximizes expected reward) imported directly into the reward function.
Abstention-aware short-form QA rewards: answering only pays off when the model is likely to be right, so the optimal policy is to answer when confident and otherwise say "I don't know" or hedge. Some prompts encourage or forbid hedging, so the user's preference for a forced guess versus a calibrated non-answer stays controllable — calibration without hedging-as-a-tic.
A dual grader for long-form instruction following: a rubric grader (checklist of what a good answer contains) paired with a claims grader that extracts each factual claim and verifies it by agentic web search, penalizing claims that don't check out.

The dual-grader design is a reward-hacking patch#

TML names the failure mode outright: rubrics "emphasize recall and can be hacked by models spraying plausibly relevant facts hoping to match rubric items." That is Reward Hacking in the instruction-following reward channel — Goodharting a checklist. The claims grader is the counter-pressure: every sprayed fact becomes a liability, because unverifiable claims are penalized individually. TML's claim is that the pair improves helpfulness and reduces hallucination simultaneously, where either grader alone trades one for the other.

The claims grader is also a deployed answer to Reference-Free Judge Over-Crediting: it does not rely on the judge model's own knowledge to score factuality — the exact condition under which reference-free LLM-as-a-Judge evaluation over-credits fluent wrong answers — but grounds each verification in retrieved evidence. The over-crediting result predicts that a knowledge-only claims grader would reward confident confabulation; putting search in the reward loop is the mitigation shape that result implies, moved from eval time into training time.

Forecasting as the resulting evaluation#

If calibration is trained on resolved questions, forecasting benchmarks are its natural exam: integrating conflicting sources into a probability, scored by Brier-type rules where overconfidence is punished symmetrically with underconfidence. TML reports Inkling at 61.1 ForecastBench Brier Index (no search) — level with Gemini 3.1 Pro (61.1) and Grok 4.3 (61.7), above GPT-5.5 (59.1) and well above Claude Opus 4.8 (54.6) — and 0.1617 on Prophet Arena, within noise of the closed frontier. All vendor-claim, on a pre-release checkpoint, tested June 30–July 13, 2026. Taken at face value, the striking part is not the win but the parity: a 41B-active open model matching frontier closed models on calibration while trailing them badly on hard reasoning (HLE 29.7 vs 44–53) suggests calibration is cheap relative to capability — a property of reward design more than of scale.

What the benchmark's own operator reports (2026-07)#

The exam named above now has a primary source, and it changes what a number on it means. ForecastBench's operator (Forecasting Research Institute, 2026-07-16, empirical) reports that several tournament submissions are statistically indistinguishable from the superforecaster median, with one above it on market questions. Three things follow for this page.

The vendor number and the leaderboard number are not the same measurement. TML's figures are a bare model with no search, answering from weights. The tournament leaderboard "is intended to capture the frontier of LLM forecasting ability, so submissions may make use of tools, added context, fine-tuning, ensembling, or other methods" — and the system at the top is a multi-stage pipeline that generates sub-questions and search queries, retrieves current context, filters for relevance and recency, ensembles models, then has another model analyze the reasoning. The frontier of this benchmark is system calibration, not model calibration. Any claim of the form "model X is at the forecasting frontier" needs to say which of the two it means; TML's table and this leaderboard sit on opposite sides of that line.

It supplies the human anchor the vendor table lacked. The superforecaster median scores 63.7 on dataset questions (N=521) and 75.9 on market questions (N=56). TML's quoted figures (Inkling 61.1, Gemini 3.1 Pro 61.1, Grok 4.3 61.7, GPT-5.5 59.1, Opus 4.8 54.6) fall below that dataset figure — if the two are the same index, which cannot be confirmed from either source: TML says "Brier Index" and the leaderboard's columns are labelled only "Dataset / Market / Overall (N)". Treat the band as suggestive, not as a placement.

And the training target is the metric's own family. RL against proper scoring rules on resolved real-world questions is, structurally, training on the thing ForecastBench measures — which is what makes "favorable ground" a live worry rather than a rhetorical one, and what makes the pipeline result above interesting: the top submissions get there by retrieval and ensembling rather than by having been RL'd toward the scoring rule at all.

The mechanism underneath the recipe, and what it does and does not say about this one (2026-08)#

The recipe above is stated as a design; Che, Yuan, Zhao & Yu (arXiv 2608.00301, 2026-07-31, empirical) supply the training-dynamics theory of the design surface it sits on, with a proof, a decay law, and live RL runs at 1.5B/7B. Their result is that error-penalized abstention rewards — +1 correct, −λ wrong, 0 abstain, the rule whose rational optimum is Chow's threshold t* = λ/(1+λ) — can drive a KL-anchored gradient learner to refuse everything while its logged mean reward rises like 1/t, because when abstention is a discrete action the reward gradient and the KL anchor's restoring force carry the same saturation factor σ′(v) and die together. Full mechanism, conditions, estimator analysis and repair on Error-Penalized Abstention Training. Three things follow for this page, and they do not all point the same way.

The paper's prescribed repair is this recipe's item 1. Their fix for the collapse is: answer every question, train a mandatory confidence report under a strictly proper score, and threshold only at deployment — because a report channel has no gate to saturate, so no shared factor can kill its reward gradient and its anchor together, and its calibrated optimum is interior and attracting (Proposition 8). That is structurally TML's proper-scoring-rule RL. The paper cites Damani et al. (2025) as the closest published relative and supplies "the missing half" of why that shape works. So the strongest reading of this source is not a criticism of the recipe — it is an independent derivation of why the recipe's continuous-report half is the right channel, and the corpus's first mechanistic argument that proper-scoring-rule RL on a report is safer than reward-shaping an abstain decision.

Two conditions the recipe's published description does not let us check. The collapse needs three ingredients jointly: (B1) blanket answering loses score in expectation on the training mixture; a shared bounded readout transmitting that loss to profitable prompts; and abstention being a discrete action. TML's item 2 says the model should "answer when confident and otherwise say 'I don't know' or hedge" — a free-form refusal with no designated decision position, which is precisely the case the paper marks untested: every one of its live collapses installs a binary first-token gate in the model's own vocabulary, and Theorem 2's proven cases exclude the generic shared-decoder shape a free-form refusal would occupy. Likewise (B1) is a property of TML's undisclosed training mixture, and the paper notes that curated or retrieval-augmented data "plausibly violate" it. So this paper does not show, or claim, that TML's recipe collapses. What it establishes is that the channel geometry of an abstention reward is load-bearing and that TML's release does not report the two facts (decision-position design, mixture score under blanket answering) that would decide it.

Two transfers that are sharp, and both are about the parts TML did not specify. First, the weights are not free (Proposition 9): scoring α·1{correct} + Brier, pure properness (α = 0) makes the calibrated accuracy gradient 2q − 1 < 0 for every q < 1/2, so the model gains score by degrading accuracy on hard prompts and honestly reporting the degradation — on exactly the prompts the loss condition selects. Any α > 1 fixes it; α = 1 leaves a degenerate stationary point at zero. TML publishes no relative weighting of its scoring-rule and correctness channels, and this says the ordering between them is a correctness condition rather than a tuning knob. Second, properness is not the operative variable (Remark 1): clip a strictly proper Brier signal below t* and the dead zone returns and deployment utility gets weakly worse while the rule stays proper. What matters is whether the training signal has a reachable region of identically zero gradient — which reframes the dual-grader discussion below, because the paper bounds its own properness claims to programmatic grading: with an LLM judge crediting hedged text at rate ν, properness degrades by O(ν). A rubric-and-claims grader is exactly such a judge, and ν is unmeasured anywhere. Hedging that a grader quietly credits is the concrete way this recipe's proper scoring rule could stop being proper.

Weight all of it against the scale gap: 1.5B and 7B Qwen2.5 on TriviaQA and PopQA, against the 41B-active model this page's numbers describe, with no long-form or agentic setting touched.

Relation to honesty-as-alignment#

Agentic Honesty & Diligence tracks the same target from the behavioral-audit side: Anthropic's overconfidence evals (admit ignorance about an unseen CLI tool; validate a teammate's wrong example) measure whether calibration shows up in agentic behavior, and Opus 4.8's constitution names "calibrated" as a component of honesty. This page is the training-loop complement: where the audit view asks does the model behave calibratedly, the TML recipe specifies what gradient produces that behavior. The two views meet at an awkward datapoint — the model whose constitution demands calibration scores lowest of the compared frontier models on TML's forecasting table — though a competitor's benchmark of a rival's checkpoint is the weakest possible evidence for that gap.

Connections#

User Awareness — the input this recipe does not control. A trained self-estimate is only as good as the invariances it holds, and the model's stated probability of solving a hard problem moves with who it believes is asking — 1.5pp at group level, 2.81 population standard deviations for the strongest identity — on questions it does not attempt
Confident But Unsure — the failure this recipe targets, measured in the wild: Opus 5 states answers its own reasoning cannot support, with accuracy up 11% and hallucination up 6% as its abstention rate falls
Inkling — the model this recipe shipped in
Reward Hacking — rubric fact-spraying is the named instance; the claims grader is the counter-pressure
Reference-Free Judge Over-Crediting — why the claims grader must search rather than rely on parametric knowledge
LLM-as-a-Judge — both graders are judges inside the reward loop; this is judge design under adversarial (Goodhart) pressure
Agentic Honesty & Diligence — the behavioral-audit view of the same target; calibration as a constitutional dimension vs. as a reward function
Thinking Machines Lab — the lab; ties to their forecasting fine-tune work on Tinker
Measuring Beyond Accuracy Saturation — where ForecastBench is treated as an instrument rather than a scoreboard: contamination-proof and non-saturating in score, yet losing discriminating power at the human reference class its headline comparison is built on
Introspective Coupling — the other trained metacognitive report in this wiki: explanation training rather than calibration training, transferring to behavior it never received labels for, and paying a comparable general-capability toll (TruthfulQA −6.7pp, GSM8K ~−5pp)
Error-Penalized Abstention Training — the training-dynamics theory of the surface this recipe sits on: why a proper score on a continuous report is the structurally safe channel and a gated abstain action is not, plus the two conditions (decision-position design, mixture score under blanket answering) this recipe's release does not report
Reasoning–Acting Interleaving (ReAct) — what the alternative looked like. Asked in late 2025 how an agent should decide when to search, CS329A lecture 4 treats calibration as unavailable ("a research problem that is still being solved") and routes around it with unconditional tool use; this page is the recipe that makes the gate buildable instead
Retrieval Inside the Reasoning Chain — where the late-2025 teaching-grade version of this agenda is recorded. Asked whether output probabilities track correctness, CS329A lecture 7 reports overconfidence and points at exactly this branch without a citation: second-pass confidence estimation, and RL or RLHF aimed at getting the model to not 'emit answers that it has low confidence over'

Open Questions#

Does abstention-aware training on short-form QA transfer to calibrated long-form and agentic self-reports (the setting where Agentic Honesty & Diligence shows overconfidence actually bites)? Partially answered (2026-08-12) — and what moved is the question, not the answer. Che, Yuan, Zhao & Yu show that "abstention-aware training" is not one object: a reward shaping a continuous confidence report and a reward shaping a discrete abstain action have different gradient geometry, and only the second has a reachable zero-gradient region where the reward signal and the KL anchor die together. So transfer has to be asked per channel, and a recipe that mixes both (as the one on this page does) can transfer on one half and fail on the other. What is not answered: the paper's live evidence is 1.5B/7B Qwen2.5 on TriviaQA and PopQA with training mixtures deliberately constructed so blanket answering loses score — no long-form, no agentic, no multi-turn, and its own report channel is read out through a linear probe on the prompt state rather than through the model's words, with the verbalized channel explicitly untested. It supplies a mechanism for why short-form results might not carry, and tests transfer nowhere. The settling experiment is unchanged in shape and now has a required control: run the same objective on long-form or agentic self-reports, reporting whether abstention is gated at a designated decision position or emitted free-form.
The claims grader verifies via agentic web search — importing the search index's coverage, recency, and bias into the reward signal. What does the model learn to claim about facts the index settles wrongly or not at all?
If calibration parity is achievable at 41B active while reasoning is not, is calibration systematically under-trained in frontier closed models (Opus 4.8's 54.6), or is TML's forecasting table just favorable ground for a model trained on proper scoring rules? Partially answered (2026-08-12) — the dichotomy is missing a third option. ForecastBench's operator shows the benchmark's frontier is held by scaffolded pipelines (retrieval + filtering + ensembling), not by bare models answering from weights, and its tournament rules permit exactly that. So a no-search vendor table and the public leaderboard measure different objects, and the Inkling-vs-Opus-4.8 gap is a bare-model comparison at a point the frontier has already left. Favorable ground is confirmed in one specific sense — the training target and the metric are the same proper-scoring-rule family — while the under-training question is untouched, since no frontier lab has submitted a bare model on a stated protocol. The settling experiment is now nameable: run the same closed models on ForecastBench's question sets with search off and search on, and compare the delta against the pipeline submissions.

Sources#

User awareness in frontier models — Zhong, Raghunathan, Laidlaw & Steinhardt, Transluce, 2026-08-06 (empirical): the uncontrolled input under a trained self-estimate: stated probability of solving a hard problem moves with who the model believes is asking (−1.5pp group, −2.81 population SDs for the strongest identity). Full treatment on User Awareness
Inkling: Our Open-Weights Model — "Epistemics" section (vendor-claim): proper-scoring-rule RL, abstention-aware rewards, rubric+claims dual grader, ForecastBench/Prophet Arena tables
Abstention as an Action Can Kill Both the Reward Gradient and the KL Anchor: Collapse Law and Repair for Error-Penalized Reinforcement Learning — Che, Yuan, Zhao & Yu (UNC Charlotte / George Mason / Auburn / North Texas), arXiv 2608.00301, v1 2026-07-31, empirical: the training-dynamics theory of this recipe's design surface. §2–3 (the three joint collapse conditions and the shared-throttling identity), §4 Propositions 8–9 and Remark 1 (why a mandatory report under a proper score is the safe channel, why the correctness weight must strictly exceed the properness weight, and why properness itself is not the operative variable), §7 limitations (free-form refusal untested; verbalized channel untested; the O(ν) LLM-judge degradation that bounds every properness claim to programmatic grading). This source never examines TML's published recipe — the transfer analysis above is this wiki's, stated as such. Full treatment and parse notes on Error-Penalized Abstention Training
AI models have likely reached parity with superforecasters on ForecastBench — Forecasting Research Institute (Substack, 2026-07-16, empirical): the tournament leaderboard's stated admission of tools/context/fine-tuning/ensembling, Cassi's pipeline description, the superforecaster median's dataset (63.7, N=521) and market (75.9, N=56) scores, and the 2024 baseline caveat. Leaderboards are PNG screenshots re-read at zoom under the image two-pass rule; the metric is never named in the post's body, so no cross-source score placement is asserted here