Fine-tuning a 230M model to know when you've stopped talking

The problem

A voice agent listening over a live microphone has to decide, continuously, whether the person talking is finished or just pausing — end-of-turn detection. The naive answer is a fixed silence timeout: stop listening after N milliseconds of quiet. It always works and always costs the full window, so short pauses within a sentence read as the end of one, and the agent either waits too long after real endings or barges in during a breath. A semantic endpointer replaces the timer with a model: read the words as they stream in and judge, from what was actually said, whether the sentence sounds finished.

Our first endpointer was a 5.26M-parameter ternary transformer, trained from scratch, running on the ANE at about 1 ms per call. It worked well on clean, fully-punctuated text and did not work on the live partial transcripts it actually had to read: the speech recognizer’s live output runs one to two words behind its own finalized transcript, so the model was being trained on a target — the finished, correctly-punctuated sentence — that never actually appears in the stream it sees at inference time. Fixing that label mismatch closed part of the gap, but the ceiling stayed low. Three separate retrains, at different real-data ratios and learning rates, all converged to the same 28–34% recall on real speech. A model trained from nothing but this one narrow task’s data doesn’t know what a finished sentence sounds like in general; it only knows what its training set happened to show it.

What changed the approach was LiveKit’s turn detector. LiveKit builds the infrastructure a lot of voice agents run on, and their own end-of-turn model isn’t trained from scratch either — it’s a fine-tune of Qwen2.5-0.5B, a small but genuinely pretrained language model, adapted to the specific decision of “has this speaker finished talking.” The model already knows what English sentences look like before it ever sees a single turn-taking example; fine-tuning only has to teach it the decision boundary, not the language. That’s the difference our from-scratch model was missing, and it’s why the fix here wasn’t a bigger version of the same architecture — it was swapping the architecture for a small pretrained model and fine-tuning that instead.

Method

1. Real partial traces. A CLI replays recorded dictation through the same streaming speech recognizer the app uses live, capturing the exact partial sequence it emits — not the clean final sentence.
2. The label fix. Across the corpus, 54% of utterances have their last live partial match the final transcript exactly; the rest lag by 1–4 words (90th percentile: 2). The original scheme labeled only the exact final transcript as a positive “done” example — unobservable live. Any partial within two words of the true end now counts too.
3. Fine-tune instead of training from scratch. LoRA rank 16 on LFM2.5-230M, framed as next-token prediction: read the transcript, predict yes or no. 3.9M trainable parameters (1.7%), real-partial and clean-text data mixed roughly 50/50.

Model and training

LFM2.5-230M: 14 layers, 8 gated short-convolution blocks and 6 grouped-query-attention blocks, 1024 hidden dim, 65,536-token vocabulary. Liquid AI picked that split via hardware-in-the-loop architecture search rather than a hand-set ratio, and distills the checkpoint from its own 350M sibling rather than training from raw text at this size. It ships native tool-call tokens in its chat template, so “read structured input, emit one constrained token” is close to its native shape.

Results

Replayed against every recorded utterance in the corpus — not a held-out slice, the full set, checking not just whether the model fires but whether it fires at the right partial:

Threshold (0.2) was picked by sweeping against the fired-at-the-right-moment metric directly. A separate 61-case synthetic set — sentences and fragments outside the recording corpus, across topics and cut after conjunctions, prepositions, articles, mid-list — scored 96.7% (59/61), complete sentences at 0.8–0.99 confidence and cut-offs under 0.05. The two misses were bare single words (“hello”, “okay”), genuinely ambiguous out of context.

One base, several tasks

LFM2.5-230M already ran two other jobs in this app — tool-call generation and chat responses — each its own LoRA adapter (7–15 MB) hot-swapped on one resident base rather than separate model copies. Turn detection is a third adapter on the same base: no new process, no new resident weights, disk cost is the adapter only. The trade is latency — a 230M forward pass is 10–30 ms uncached, not the ~1 ms the old 5M model cost, so it’s called on a timer rather than every audio chunk.

Small and built for structured output

Liquid AI searches the conv/attention split against real hardware latency and memory constraints rather than scaling a transformer down, and distills the 230M checkpoint from its own 350M sibling rather than training it from raw text at that size. It ships native tool-call tokens in its chat template — consistent with it holding up on structured-extraction and tool-use benchmarks well for its size.

How Haku uses it

Additive and fail-safe: confident, it ends the turn early; unsure, the existing silence timeout ends it exactly as before. Runs on-device, in-process, no network round trip for the decision.

Links

LFM2 technical report — arxiv.org/abs/2511.23404
LiquidAI/LFM2.5-230M — huggingface.co/LiquidAI/LFM2.5-230M
LiveKit turn detector — livekit.com/blog/solving-end-of-turn-detection