Can You Trust the Probability? Testing a Decision Model on 77 PubMed Papers
A decision model's probability is not how sure it is. On 77 PubMed papers, see how to check calibration and set a threshold from coverage and accuracy.
A decision model's probability is not how sure it is. On 77 PubMed papers, see how to check calibration and set a threshold from coverage and accuracy.
A decision model reads the logits of listed options after one forward pass and turns them into probabilities. Jev discloses RLCD but not its loss; open replicas publish their heads.
This is an AI memory system we designed ourselves from a year of actual use. It does not use a vector database. It uses plain-text files plus a trigger-phrase index. Six figures show how it works: three roles, trigger phrases written as symptoms, the startup index character budget, the propose → judge → apply flow for merging memory, a hit-rate measurement over 301 sessions, and the trade-offs of this approach.
Several AI CLIs shared a memory system we designed ourselves, in one workspace, for a year. It accumulated about a thousand episodic memories and more than five hundred knowledge notes. What caused the loss was not that the AI forgot. It was that the AI trusted something it had not checked. This piece covers the three fixes in Ghost In Shell 5.2, and six pitfalls we hit ourselves, each with the check we use now.
On one RTX 4060 Ti 16GB and one frozen set of 700 items, Ridge, OrcaSAQ-2, and Ternary Bonsai 2 scored 78.4%, 77.6%, and 74.6%. Wall clocks were 47.55, 62.59, and 34.01 minutes. This compares three deployments, not one set of weights.
A decision model returns a probability for every pre-listed option in one forward pass and writes no text. Here is how it differs from LLM JSON output, where Jev and Clef stand, and when not to use one.
On an RTX 4060 Ti 16GB seat, Prism Ternary Bonsai 2 27B (PQ2_0) was scored against a same-box Qwen3.8 dense daily model across short functional gates, long-context needles, and tool loops before a stable default alias pointed at the ternary engine.
DeepMind’s AlphaGenome Atlas precomputes molecular effects for about nine billion possible single-letter DNA changes and ranks them with AVI scores. A research traffic map, not a clinical diagnosis.
How to turn manual copy-pasting across CLI agent terminals into auditable mutual task assignment using dynamic discovery, capacity allocation, and file-first handoffs.
From acceleration knobs and extreme quantization to a nine-model comparison, a large-sample retest at month's end reshuffled the entire leaderboard. A record of a broken yardstick, small-sample error bars, and the current division of labor between pi and local models.