Haku Lab
Small language models that run on your Mac, not in the cloud.
Haku is a tiny lab. We research small language models that run intelligently on-device — on the Mac for now, across its GPU and Neural Engine — and we build the products around them. The bet is simple: for a lot of real work you don't need a giant model behind an API. A model small enough to fit in a few megabytes, quantized to ternary or low-bit weights, can do one narrow job well — on the device, with no network in the loop, in milliseconds.
What we do
- Train tiny specialists. From-scratch models for one narrow job — grammar, dates, typing web elements, end-of-turn detection — each small enough that a mesh of them fits within a phone's budget.
- Quantize to the metal. Ternary and low-bit weights via quantization-aware training, so a model fits in megabytes and runs on the Apple Neural Engine at little or no accuracy cost.
- One base, many adapters. A frozen ternary base plus small task-specific LoRA adapters — new capabilities without retraining, fine-tuned cheaply on the Mac itself.
- On-device, end to end. Inference runs on the GPU and Neural Engine through our own Metal engine. Your data stays on your machine.
Experiments
Short, honest write-ups — what we tried and what we got, with the numbers.
Indexing code by parsing it, not chunking it
Haku's code search reads each language's syntax tree, indexes real symbols, and embeds them on the Neural Engine — running inside the terminal, nothing leaving the Mac.
Read →A 9M ternary encoder for accessibility-tree elements
One layer of the filter between raw accessibility data and the agent — element routing, on the Neural Engine.
Read →Resolving natural-language dates with a tiny specialist model
Division of labour — a 14M model maps language to structure; ordinary code does the arithmetic
Read →An experiment to format dictation in under 100 milliseconds
A 60M-parameter T5, distilled from a larger model, that cleans up raw speech on-device — split across the Neural Engine and the GPU.
Read →Grammar correction with a diffusion LoRA on a frozen model
A small adapter turns a frozen left-to-right model into a one-pass, both-directions corrector.
Read →An experiment to cut down TTS latency
Making a text-to-speech model decode in one pass instead of sixteen, so it runs fast on-device. With audio you can play.
Read & listen →Parallel decoding of audio tokens in a text-to-speech model
Decoding 16 residual codebooks in one pass instead of sixteen. On-device.
Read →Diffusion-draft speculative decoding: an eight-prompt probe
A diffusion model drafts, an autoregressive model verifies. Acceptance measured on eight prompts.
Read →Structured tool-calls from a 25M model
How small can a model be and still learn a structured capability?
Read →A few months of ternary experiments
Where ternary {−1, 0, +1} weights are the right tool for small, on-device models — and where they aren't.
Read →