Llmspeed
What it is
Pick a GPU or Mac and a model, and it plays tokens back at the speed you'd actually get.
Featured in
Issue #18 — Spare Keys · — the full issue has that week's other tools, reads, and big-lab releases.
More local AI tools we've covered
- Janas A C engine that streams mixture-of-experts weights from SSD, so a 48 GB, 80B model runs on a 32 GB laptop.
- Magnitude Local inference engine for coding agents that tunes its kernels to your exact hardware before running a model.
- Cactus Needle 3 A foundation model for phones, wearables, and microcontrollers, built for automation rather than chat: tool calls, structured extraction, and embeddings. The trick is an 'intelligence ladder' — one set of weights where every depth from 2 to 20 layers is a usable model of its own, shipped as 2-bit binaries between roughly 9 and 29 MB — so you choose accuracy versus footprint per device without training a family. Open weights on Hugging Face under Apache-2.0; Cactus sells a one-off $19 hosted fine-tune.
- Desert Ant Labs A new European lab, launched this week by Paul Veugen, shipping eighteen specialised on-device models for audio, vision, and text: transcription it claims runs 4.7× faster than Whisper, audio enhancement in 9MB, PII redaction across 27 languages in 12MB, language identification in 2MB. Everything runs on the phone's neural engine or in-browser WASM; Swift, Kotlin, and JS SDKs; free up to 100,000 monthly active devices, no tokens, no logins.