Indexing code by parsing it, not chunking it

Haku's code search reads each language's syntax tree, indexes real symbols, and embeds them on the Neural Engine — running inside the terminal, nothing leaving the Mac.

A common, easy way to add semantic search to code is to split files into fixed-size text windows and embed those. We parse instead. Haku runs a real parser for each language, walks the syntax tree, and emits one index entry per symbol — a function, a type, a method — carrying its name, its leading doc comment, and its body. The symbol's surface — its name and doc — is embedded with EmbeddingGemma-300M on the Apple Neural Engine and searched by meaning. The result is that "handle websocket reconnection" finds the function that does that, even when those words appear nowhere in it — and the whole index lives, runs, and stays inside the terminal on your machine.

Scope. This describes the version that ships in HakuTerminal, where the search engine is built into the app and the embedder runs on the Neural Engine via Core ML. Measured: ~18 ms per text embedding with 99.6% of operations resident on the ANE, and a mean cosine of 0.984 against the reference (non-quantized) embeddings — i.e. the on-device path matches the reference model closely. Model-load time is paid once, lazily, at app launch; it is logged but not separately benchmarked, so it isn’t quoted here.

Why not just chunk the text

A common move for “add semantic search to code” is to slide a fixed window over each file — say 1,200 characters with some overlap — and embed each window. It is easy, language-agnostic, and wrong for code in three ways. A window cuts functions in half, so a query matches a fragment with no beginning or end. It mixes unrelated code that happens to be adjacent. And it throws away the single most useful signal a code unit has: its name. A function called reconnectWebSocket is a strong hint about what the code does, and windowing dilutes it into the middle of a blob.

We index the unit a programmer actually thinks in — the symbol — and we get its name for free, because to find the symbols you have to parse the language anyway.

A parser per language

For each of ten languages the indexer loads the matching tree-sitter grammar and parses the file into a real syntax tree:

RustSwiftPythonJavaScriptTypeScriptTSXGoJavaRubyC

A language-specific walker then descends the tree and emits a symbol at each node that names a unit of code. For Rust that’s function_item, struct_item, enum_item, trait_item, impl_item, and so on; for the C-family and scripting languages a small config drives a generic walker over function_declaration, method_definition, class_declaration, and their kin. The walker threads the enclosing type or trait down as it recurses, so a method comes out qualified — DOMClassifier.predict, not a bare predict floating free of its class.

Each emitted symbol is a small record:

struct Symbol {
    name,          // reconnectWebSocket
    natural_name,  // "reconnect web socket"  (see below)
    file, line, project,
    doc,           // the leading /// or /** comment, capped 200 chars
    body,          // the symbol's source, capped 30 lines / 1500 chars
}

Parsing also lets us skip what shouldn’t be indexed at all — node_modules, target, build, vendor, .venv, and the like are pruned before a single file is read.

The name is a feature: natural-name expansion

Identifiers are written for compilers, not for search. JSONResponseParser is one token to an embedder, and a query like “parse json” won’t land near it. So at index time every identifier is split on case and separator boundaries into the words it’s made of:

JSONResponseParser  →  "json response parser"
index_folder        →  "index folder"

The split handles the awkward cases — the acronym-to-word boundary in JSONResponse, snake_case, kebab-case, dotted and colon-separated paths.

This expansion matters more than it looks, because of what actually gets embedded. The text fed to the model for each symbol is just:

"{project}: {name} {natural_name} {doc}"

— the symbol’s surface, not its body. We embed what the code announces about itself — its name, the words that name decomposes into, and its doc comment — and deliberately leave the implementation out of the vector. The body is still stored, and still returned with a hit so you can read it, but it doesn’t drive the match. That keeps each vector about one clear idea (a blob of implementation tends to smear the embedding toward whatever it happens to call), and it puts almost the entire matching burden on the name — which is exactly why expanding handleWebSocket into “handle web socket” is what makes “handle websocket reconnection” land.

Embedding on the Neural Engine

Each symbol’s text is embedded into a 768-dimension, L2-normalized vector by EmbeddingGemma-300M, tokenized with the standard tokenizer at a fixed 512-token length and run through Core ML on the Apple Neural Engine. This is the same reason the rest of Haku targets the ANE: it is fast, power-efficient, and — being a local model — it means your source code is never sent anywhere to be indexed or searched. A query is embedded by the same model, so query and code live in one space and a plain cosine compares them.

Storage and search

The index is a SQLite database, one row per symbol: its metadata — project, file, line, name, expanded name, doc, body, a content hash — and its embedding stored as a blob. SQLite is the source of truth. Search runs against a compact sidecar built from those embeddings: a 4-bit quantized vector index (the turbovec crate — a .tvim file keyed by row id), which keeps the resident vectors at roughly an eighth the size of full-precision f32. A query is embedded by the same model, scored against the sidecar, and the top-k row ids are returned, with an optional allowlist to scope a search to specific projects. The cost that matters per query is the one embedding (~18 ms on the ANE).

Ingest is two-phase and resumable, so it never blocks the terminal. Phase one parses the files and writes the symbol rows with a null embedding; a background worker then pulls the pending rows, embeds them, and fills in the blob — running at lowered priority so it never competes with drawing the terminal. Re-indexing is incremental: a per-symbol content hash detects what actually changed, unchanged rows keep their embedding, and only new or modified symbols are re-embedded. The sidecar is rebuilt from the stored embeddings when the set changes.

Productionized inside the terminal

This isn’t a library you call; it’s a service that’s always on. It ships inside HakuTerminal, and the terminal process is the index server — there’s no separate daemon to launch or socket to manage. Three things make it feel live:

It watches your code. A background thread watches every pinned project for file changes, debounces a burst for two seconds, and re-indexes only what moved. The thread runs at lowered priority so indexing never competes with drawing the terminal.
It reaches remote machines. A project living on another box over SSH is indexed by tarring the source, pulling it across, and indexing it locally — so search spans the machines you actually work on, not just this one.
It’s built for an agent. The code_search tool takes a batch of queries in one call, runs each, and merges: a symbol found by more than one query is deduplicated (keyed by project-file-line-name) and tagged with the queries that surfaced it (matched_queries). An agent surveying a subsystem asks five questions at once and gets one attributed, deduplicated list — fewer round-trips, and a signal of which hits are central.

What this is and isn’t

The embedder is an off-the-shelf 300M model, not a bespoke tiny one.
Symbol- and function-level indexing isn’t novel either — function units are standard in code search (e.g. CodeSearchNet), and some tools chunk along the syntax tree rather than by fixed windows. What’s specific here is the combination: embedding the symbol’s surface (name, expanded name, doc) rather than its body, name expansion, and an always-on on-device service that runs the embedder on the Neural Engine inside the terminal.
Ten languages are parsed precisely; everything else is simply not indexed (there is no text-window fallback for unknown languages). That’s a deliberate trade of coverage for precision.
The numbers quoted are measured on the Neural-Engine build: ~18 ms per embedding, 99.6% ANE residency, 0.984 cosine against the reference embeddings. End-to-end query latency (embed + scan) and model-load time are not separately benchmarked here, so they aren’t quoted.

Links

turbovec (4-bit vector quantization + SIMD search) — github.com/RyanCodrai/turbovec