From acceleration knobs and extreme quantization to a nine-model comparison, a large-sample retest at month's end reshuffled the entire leaderboard. A record of a broken yardstick, small-sample error bars, and the current division of labor between pi and local models.
I benchmarked five 4-bit local models on a single M4 Pro (48GB) with oMLX across eight benchmarks, mapped the speed/quality sweet spot, and calibrated which cloud generation they match. Plus a field report on three speculative-decoding traps (MTP / dflash / specprefill) and the cache trade-off that decides them.
Auto-Quant lets LLMs autonomously iterate on FreqTrade strategies via backtesting loops. Interesting autoresearch concept, but deeply locked into crypto trading infrastructure. Only useful if you live in the FreqTrade ecosystem.
Starting from the everyday experience of watching ChatGPT type out words one by one, this article uses six everyday analogies to break down KV Cache, Paged Attention, Flash Attention, GQA, Continuous Batching, and Speculative Decoding — explaining how large language models serve more users faster on the same GPU.
Feeding 47 papers into Claude costs $9.59 per 5-round session. Using NotebookLM for retrieval and Claude for reasoning drops that to $0.55. The secret: not a single word of your papers ever enters Claude.
