Haku Lab

Tag

MoE

← all experiments · all tags

2026Ternary · slice adapters · LoRA · ANE

Slice adapters vs LoRA on a ternary model

Retrain one slice of a ternary model's feed-forward channels in place, instead of adding a LoRA side path. Compared on four ternary models, timed on the M1 Neural Engine.

Read →
20267 failures · 6 models · no gradients, no training

Fixing small models by editing their weights: 5 of 7 failures

An MIT team fixed a looping model by editing its weights, with no training. We tried the same idea on 7 failures in 6 of our small models, with a random edit of the same size as the control.

Read →