kimi-k3-in-c
Star of the week, issue #7What it is
A portable C99 implementation that runs the 2.78-trillion-parameter Kimi K3 on one CPU in 8.24 GB of RAM. Six C files, AVX2 and OpenMP, no GPU and no BLAS — dormant experts stream off disk while 16 of 896 fire per token.
Featured in
Issue #7 — Frontier Weights Go Public · — the full issue has that week's other tools, reads, and big-lab releases.
More local AI tools we've covered
- Llmspeed Pick a GPU or Mac and a model, and it plays tokens back at the speed you'd actually get.
- Janas A C engine that streams mixture-of-experts weights from SSD, so a 48 GB, 80B model runs on a 32 GB laptop.
- Magnitude Local inference engine for coding agents that tunes its kernels to your exact hardware before running a model.
- Cactus Needle 3 A foundation model for phones, wearables, and microcontrollers, built for automation rather than chat: tool calls, structured extraction, and embeddings. The trick is an 'intelligence ladder' — one set of weights where every depth from 2 to 20 layers is a usable model of its own, shipped as 2-bit binaries between roughly 9 and 29 MB — so you choose accuracy versus footprint per device without training a family. Open weights on Hugging Face under Apache-2.0; Cactus sells a one-off $19 hosted fine-tune.