Haku Lab
Small language models that run on your Mac, not in the cloud.
Haku is a tiny lab. We research small language models that run intelligently on-device — on the Mac for now, across its GPU and Neural Engine — and we build the products around them. The bet is simple: for a lot of real work you don't need a giant model behind an API. A model small enough to fit in a few megabytes, quantized to ternary or low-bit weights, can do one narrow job well — on the device, with no network in the loop, in milliseconds.
What we do
- Train tiny specialists. From-scratch models for one narrow job — grammar, dates, typing web elements, end-of-turn detection — each small enough that a mesh of them fits within a phone's budget.
- Quantize to the metal. Ternary and low-bit weights via quantization-aware training, so a model fits in megabytes and runs on the Apple Neural Engine at little or no accuracy cost.
- One base, many adapters. A frozen ternary base plus small task-specific LoRA adapters — new capabilities without retraining, fine-tuned cheaply on the Mac itself.
- On-device, end to end. Inference runs on the GPU and Neural Engine through our own Metal engine. Your data stays on your machine.
Experiments
Short, honest write-ups — what we tried and what we got, with the numbers.
Does the 27B Bonsai actually deliver 27B?
Prism shipped a 27B that fits a laptop. We put it through a battery of agentic traps to see if it reasons like its size.
Read →Running a ternary 27B on a single A10
A 27B model whose weights are all −1, 0 or +1 — what it takes to run, and how fast it goes.
Read →Fine-tuning a 230M model to know when you've stopped talking
A from-scratch 5M-parameter classifier plateaued at 30% recall on real speech. A LoRA fine-tune of a 230M pretrained model, on the same data, didn't.
Read →Indexing code by parsing it, not chunking it
Haku's code search reads each language's syntax tree, indexes real symbols, and embeds them on the Neural Engine — running inside the terminal, nothing leaving the Mac.
Read →Resolving natural-language dates with a tiny specialist model
Division of labour — a 14M model maps language to structure; ordinary code does the arithmetic
Read →A 9M ternary encoder for accessibility-tree elements
One layer of the filter between raw accessibility data and the agent — element routing, on the Neural Engine.
Read →An experiment to format dictation in under 100 milliseconds
A 60M-parameter T5, distilled from a larger model, that cleans up raw speech on-device — split across the Neural Engine and the GPU.
Read →Grammar correction with a diffusion LoRA on a frozen model
A small adapter turns a frozen left-to-right model into a one-pass, both-directions corrector.
Read →An experiment to cut down TTS latency
Making a text-to-speech model decode in one pass instead of sixteen, so it runs fast on-device. With audio you can play.
Read & listen →Parallel decoding of audio tokens in a text-to-speech model
Decoding 16 residual codebooks in one pass instead of sixteen. On-device.
Read →Diffusion-draft speculative decoding: an eight-prompt probe
A diffusion model drafts, an autoregressive model verifies. Acceptance measured on eight prompts.
Read →Structured tool-calls from a 25M model
How small can a model be and still learn a structured capability?
Read →A few months of ternary experiments
Where ternary {−1, 0, +1} weights are the right tool for small, on-device models — and where they aren't.
Read →