Experiment
Slice adapters vs LoRA on a ternary model
Small on-device models are usually specialized with LoRA: a frozen base plus a low-rank side path per task. On a ternary base, that side path usually stays unmerged, because merging means requantizing. We tried a different unit: a ternary slice adapter. It retrains one slice of the model's feed-forward channels in place, at ternary precision, while every other weight stays frozen. Nothing is added to the graph, so the deployed model keeps the base model's shape.
We compared slice adapters with LoRA on conversation, app commands and tool calling, on four ternary models, including our 203M-parameter ternary mixture of experts and BitNet b1.58-large. On the M1 Neural Engine the slice adapter adds no decode time; a LoRA side path adds 1.7% to 41% depending on rank. Merging a LoRA into the ternary weights loses most of its gain. On our model and on BitNet b1.58-large, slice adapters match or beat LoRA of equal size on all three tasks. This post is the short version of a paper that is not public yet.
vs base (M1 ANE)
rank 8 to 480
vs 1.889 equal-bytes LoRA
2-bit codes + scales
The idea
Our base model is a small ternary mixture of experts: 12 layers, 4 experts per layer, every projection weight −1, 0 or +1 times a scale. Its experts are already separate units. A slice adapter is one expert retrained on a task — 1,920 of the 7,680 feed-forward channels in every layer — at ternary precision, with every other parameter frozen. We hash the frozen parameters before and after training; they are byte-identical.
At run time the whole request is routed to that expert. The deployed graph is the base graph with different codes, so the slice adapter costs exactly what the base costs. Stored as 2-bit codes with one fp16 scale per 128-weight block, one slice adapter is 11,750,400 bytes (11.75 MB).
LoRA is the obvious alternative. We applied it to the same expert (the w1, w3 and w2 projections in all 12 layers). At fp16, the equal-bytes LoRA is rank 64: 11.8 MB against the slice adapter’s 11.75 MB.
Method
| Task | Train | Test | Metric |
|---|---|---|---|
| Conversation (SODA-41k) | 41,000 rows | 12,186 rows | response loss, lower is better |
| App commands | 2,200 rows, 55 intents | 440 rows by an unseen writer; 1,090 rows of outside text | intent exact; app accuracy |
| Tool calling (tc41k) | 41,580 prompts | 4,508 rows; BFCL v3 simple, 400 rows | semantic exact |
LoRA ranks 8 to 480, same expert, same data, steps and seeds. The two methods want different learning rates, so each arm was swept until its best rate was inside the grid, and each is reported at its own best rate. Our first comparison used one shared rate, and it was misleading in both directions.
Results on our model
Conversation, valid loss, mean (min–max) over seeds:
| Arm | Bytes | Seeds | Valid loss | M1 ANE decode vs base |
|---|---|---|---|---|
| Foundation, no adapter | – | 1 | 3.413 | – |
| Slice adapter | 11.75 MB | 3 | 1.835 (1.834–1.836) | −0.2% |
| LoRA r64 | 11.8 MB | 3 | 1.889 (1.888–1.889) | +3.0% |
| LoRA r128 | 23.6 MB | 3 | 1.865 (1.864–1.865) | +5.7% |
| LoRA r256 | 47.2 MB | 1 | 1.845 | +16.3% |
| LoRA r480 | 88.5 MB | 1 | 1.828 | +41.4% |
At equal bytes the slice adapter beats r64 by 0.054; seed spreads are 0.002 or less. Only r480 scores lower, on one seed, at 8× the bytes and +41.4% decode time. On an earlier full-epoch run (one seed, one shared rate), r480 won outright, 1.421 vs 1.447.
Tool calling, test split semantic exact (n=4,508), 3 seeds:
| Arm | Bytes | Test split % | BFCL simple /400 |
|---|---|---|---|
| Slice adapter | 11.75 MB | 77.05 (74.62–80.06) | 67.3 (60–71) |
| LoRA r8 | 1.5 MB | 70.37 (68.83–71.74) | 12.0 (1–31) |
| LoRA r64 | 11.8 MB | 75.22 (74.56–75.55) | 68.3 (65–71) |
| LoRA r128 | 23.6 MB | 76.63 (73.62–78.59) | 70.3 (68–72) |
| LoRA r256 | 47.2 MB | 76.59 (74.49–78.57) | 65.7 (63–69) |
Seed ranges overlap, so the row-paired test carries this one: against r64 the slice adapter is significantly better in 2 of 3 same-seed pairs (exact McNemar, p < 0.05) and tied in the third. r128 and r256, at 2–4× the bytes, are within the slice adapter’s seed range. BFCL simple is a tie between the slice adapter and r64–r256.
App commands, in-distribution intent: a tie, 96.1 vs 95.5 for r64 (n=440, 3 seeds each). On 1,090 rows of realistic outside text the slice adapter leads on app accuracy, 77.4 vs 71.4 — but r64 at a different rate reached 74.8 (71.6–77.3), which overlaps.
What a side path costs on the Neural Engine
Decode time on the M1 Neural Engine, n=6,400 decode steps per arm:
| Arm | Decode median ms | vs base | Package MB |
|---|---|---|---|
| Base | 2.615 | 0.0% | 36.5 |
| Slice adapter | 2.611 | −0.2% | 36.5 |
| LoRA r8 | 2.660 | +1.7% | 38.1 |
| LoRA r64 | 2.695 | +3.0% | 48.4 |
| LoRA r128 | 2.763 | +5.7% | 60.2 |
| LoRA r256 | 3.041 | +16.3% | 83.8 |
| LoRA r480 | 3.698 | +41.4% | 125.1 |
Random adapter weights; timing depends on shapes, not values. Trained arms were scored on the ANE separately and stayed within ±3 rows of their GPU scores in 15 of 15 cells.
Any side path costs about 1.7%, even at rank 8: the graph gains 108 ANE operations whatever the rank. The cost grows slowly up to r128 and steeply after. Moving the side path to the GPU does not help: every arm decodes fastest on the ANE.
Why not merge the LoRA?
Merging would remove the side path. We merged 18 trained LoRAs into the ternary expert and requantized. Most LoRAs move almost no ternary codes — the per-weight change is 0.04–0.12 of a ternary step — so rounding erases them. In no arm did a merged LoRA match its unmerged self or beat the slice adapter. The best case went from 1.404 unmerged to 1.702 merged, against a slice adapter at 1.646 (and that slice adapter was a weaker, low-learning-rate one).
Is this just our model?
Our base was built so one expert can be trained alone. To check that this design is not the whole story, we repeated the comparison on BitNet b1.58-large, a dense 729M ternary model. There a slice adapter is 400 of the 4,096 feed-forward channels in every layer — 44,236,800 weights, the same count as our expert — retrained in place.
| BitNet b1.58-large arm | Bytes | Conversation loss (seeds) | Tool call % (n=600, 1 seed) |
|---|---|---|---|
| Slice adapter | 11 MB | 1.2705 (3) | 66.0 |
| LoRA r8 | 6.5 MB | 1.2923 (1) | 60.2 |
| LoRA r16 | 13 MB | 1.2856 (3) | 60.3 |
| LoRA r64 | 52 MB | 1.2670 (1) | 62.7 |
The slice adapter beats the equal-bytes LoRA (r16) on conversation and tool calling, and adds no decode time on the M1 ANE (−0.014 ms, against +0.216 ms for r16). A LoRA at 4.7× the bytes (r64) matches it on conversation, on one seed. App commands tie at the ceiling (97.8 vs 97.8).
On two smaller dense ternary models of ours (99M and about 350M, with no activation quantization), the result is weaker. The slice adapter leads the equal-bytes LoRA on conversation by a small margin, on one LoRA seed, but a LoRA at 1.6–2.5× its bytes beats it. On the 99M model the equal-bytes LoRA also beats it on app commands, 89.5 vs 83.4 (one seed each; the slice adapter ran at one learning rate there). That is where the claim stops for now.
Limits
- Format brittleness. On tool-schema renderings never seen in training, the slice adapter breaks its output format more often: 49% malformed on the worst rendering (59 of 120 rows over 3 seeds) vs 12% for the LoRAs (72 of 600).
- Quantized LoRAs. Stored in 8 bits, the equal-bytes LoRA becomes r128, and the quality lead holds on conversation only. We did not train or time quantized LoRAs.
- Codes-only models. On a model released without latent weights, results were inconclusive.
- Switching. A slice adapter is not yet its own file: on the ANE it is compiled into a 36.5 MB whole-model package. Switching takes 113.8 ms (median, n=12) while that package stays at a fixed path; a new path recompiles for about 2.2 s.
Where the slice adapter wins, it wins on bytes and device cost rather than on rank. A rank-64 truncation of a slice adapter’s change keeps 99% of its gain on app commands and 81% on conversation, so a LoRA could hold most of it. It needs more bytes and a side path to do so.
Links
Paper — not public yet
How weight encoding affects placement on the Apple Neural Engine — arxiv.org/abs/2608.22110
1bitLLM/bitnet_b1_58-large — huggingface.co/1bitLLM/bitnet_b1_58-large
LoRA (Hu et al., 2021) — arxiv.org/abs/2106.09685
Get new experiments
Occasional notes when we publish. No spam.