241 Vectors: The Search Problem You Do Not Have

Two Notes, One Machine

The first note came out of a car. It is filed with a source line that describes the person as roadside at the time, and the idea is one that anyone who keeps a large pile of their own thoughts will recognise immediately.

Semantic search over the ideas. Embeddings on title and text. Pär, you had this same thought in March.

The second note came 2 months later and is about the server rather than the search. It asks whether things could sit on the machine switched off. If you want to play with a vector database, or something heavier than usual, could it be there without running, and come alive only when asked.

Put together, they describe a small, sensible, appealing weekend project. And the research produces an answer that is going to be slightly disappointing and considerably more useful than the project would have been, because the honest finding is that the hard problem is not the one either note is about.

The Number That Ruins The Fun

The capture inbox currently holds 241 open items.

That number is the entire story, and everything else follows from it, so it is worth being blunt about what it means.

Vector search exists as a discipline because comparing a query against every single item in a collection is too slow when the collection is enormous. All the interesting machinery — the graph indexes, the clustering, the quantization schemes, the specialised database products — exists to avoid doing the obvious thing. They are all approximations that trade a small amount of accuracy for an enormous amount of speed, and every one of them is worth it at scale.

At 241 items, the obvious thing takes no measurable time at all.

Consider the arithmetic. A vector in the standard format takes 4 bytes per dimension plus a small header. At 1,024 dimensions that is a little over 4 kilobytes per item. Multiply by 241 and the entire collection is under a megabyte. That is small enough to sit comfortably in a processor's cache. Comparing a query against all of it is a few hundred thousand floating point operations, which a modern processor does in the time it takes to think about doing it.

So the correct index for this collection is no index. Not a small index. Not a carefully tuned one. None. Add the column, store the numbers, and let the database compare all 241 every time, because it will finish before the network round trip does.

That remains true for a long time. It stays true at 10,000 items. It is still generally fine at 100,000, where an exact scan runs in tens of milliseconds and nobody notices. The point where the machinery starts earning its keep is somewhere in the millions, and an inbox that has accumulated 241 items over several years is not going to reach millions during any lifetime that concerns us.

What A Vector Actually Is

Since the machinery is being set aside, it is worth spending the time on what the thing underneath actually does, because that part matters at every scale.

An embedding model takes a piece of text and produces a long list of numbers, typically several hundred to a couple of thousand of them. The numbers have no individual meaning. No single one of them is "about weather" or "sounds urgent". What they have collectively is a geometry: texts that mean similar things end up close together in that space, and texts that mean different things end up far apart. The model learned this arrangement by being shown enormous quantities of text and being trained to place related passages near each other.

That is the whole trick, and it is why it feels like magic when it works. You are not searching for words. You are searching for a location. A note about pressing a button in the shower and a note about capturing thoughts while your hands are wet contain almost no words in common, and they land in nearly the same place.

Distance between 2 of those locations is computed with straightforward arithmetic, and there are a few standard ways to measure it depending on whether the model's outputs are normalised. Choosing the wrong measure for your model is one of the classic quiet failures in this area: everything runs, no errors appear, and the results are subtly worse than they should be forever.

The Real Problem Is The Words

Here is where the project gets genuinely difficult, and it has nothing to do with databases.

The inbox is not a clean corpus. It is voice notes, transcribed automatically, in a mixture of Swedish and English, often switching language inside a single sentence, recorded while driving or walking or standing in a workshop. The transcription is good but not perfect, and the errors are systematic in a way that matters here.

Go and look at the actual text. A note that plainly meant "locked out" was transcribed as "logged out". A static site generator called Zola came through as "Sola", with the speaker helpfully spelling it out letter by letter afterwards, which the transcript preserved as a stray sequence of letters. Words that are ordinary in one language get rendered as near-miss words in the other. Sentences trail off unfinished because the recording stopped.

An embedding model handles this less gracefully than people expect. It does not know that "logged out" was a mistake. It embeds what it was given, and it places the note wherever "logged out" belongs, which is somewhere near authentication and session management rather than near locked doors and walled gardens. The error is not loud. It just quietly puts the note in the wrong neighbourhood, and it will never be found by the query that should have found it.

Then there is the language question. A model trained mostly on English will embed Swedish text, but it will do it badly, and worse, it will not reliably place a Swedish sentence near an English sentence that means the same thing. For a bilingual corpus, that is disqualifying. You need a model explicitly trained across languages, which narrows the field considerably and is the single most consequential choice in the whole project.

None of that gets solved by choosing a better database, and all of it determines whether the feature works.

Near-Duplicate Is Not Search

Now the more interesting observation, which is that the note asks for 2 different features and only one of them is search.

"Find me notes about hardware buttons" is semantic search. You have a query, you want relevant results, and relevance is fuzzy and generous. Good results at the top, some noise below, and the human sorts it out.

"You had this same thought in March" is not that. That is near-duplicate detection, and it has completely different requirements. It runs at write time rather than read time. It has no human query, because the new note is the query. And it only needs to fire when the similarity is very high, because a notification that says "this is a bit like something you wrote once" is worse than no notification at all.

That distinction changes the design in the person's favour. Near-duplicate detection over 241 items is trivially cheap, because it runs once per new note and compares against everything, which is one pass over a megabyte. It needs no interface, because its output is a line appended to the new item saying what it resembles. And it is much more forgiving of a mediocre embedding model, because you are only acting on the strongest matches, and the strongest matches are the ones every model gets right.

It is also the feature that actually pays. An inbox with 241 open items in it almost certainly contains the same idea recorded 3 times, months apart, in slightly different words, by someone who had genuinely forgotten. Finding those is not a nice-to-have. It is a measurement of how much of the pile is real.

The Keyword Index Is Already There

There is a second thing sitting in this database that most vector projects forget to use, and in this case it is already built.

The items already have full text search on them, configured to treat the text simply and without language-specific stemming, which is the correct choice for a mixed Swedish and English corpus where an English stemmer would mangle the Swedish. The coding sessions have their own search, configured differently, with English stemming and accent folding applied consistently on both the writing and the querying side.

Keyword search and vector search fail in opposite directions, and that is what makes them worth combining. Keyword search is precise and literal: it will find the note that says "Zola" and it will completely miss the note that describes the same idea without naming it. Vector search is associative and vague: it will find the conceptually related note and it will sometimes fail to find the exact one containing the exact word, because a single distinctive term does not move the geometry much.

The standard way to combine them does not involve tuning weights or normalising scores from 2 different scales, which is a trap people fall into and never escape. Instead you run both searches, take the rank position of each result in each list, and combine those positions with a simple formula that rewards appearing high in either list. An item ranked second by keywords and eighth by meaning beats an item ranked fortieth by both. It requires no calibration, it is a few lines of query, and it consistently outperforms either method alone.

Given that half of this is already built, the honest scope of the project is: add a column, compute embeddings, and fuse 2 rank lists. That is an evening, not a weekend.

The Machinery, And When It Would Matter

For completeness, and because the note deserves to know what it is declining, here is what all the excitement in this area is actually about.

There are 2 main index types. One clusters vectors into groups so a query only searches nearby groups. The other builds a navigable graph in layers, where the top layer is sparse and lets you jump across the space, and each layer down is denser, so a search descends toward the right neighbourhood. The graph approach dominates in practice because it gives better recall for the same speed, at the cost of slower building and more memory.

Both are approximate. They do not promise to find the true nearest match. They promise to find a very good one, very fast. Recent versions added a way for the planner to keep scanning further into the index when a filter has thrown away most of the candidates, which fixed a genuinely nasty failure where a filtered query would come back empty despite matching rows existing.

Then there is quantization, which is the elegant part. A half-precision format halves the storage with almost no loss of accuracy, and doubles how many dimensions can be indexed, because the limits come from how much has to fit in a database page. Binary quantization goes much further: reduce every dimension to a single bit depending on its sign. A 1,024 dimension vector drops from just over 4 kilobytes to 136 bytes, a reduction of about 30 times, and comparing 2 of them becomes a bitwise operation the processor does natively at enormous speed. You then re-rank the top candidates using the full vectors to recover the accuracy.

That works startlingly well, but with a caveat that is easy to miss: it works best with embedding models specifically trained to survive binary compression. Applying it to a model that was not is a good way to lose quality without understanding why.

And there is an extension that pushes the ceiling further with its own algorithm and compression scheme, which narrows the gap between a general database and a purpose-built vector engine considerably.

All of this is genuinely good engineering. None of it applies below a million vectors. Knowing it exists is worth 10 minutes so that you recognise the day you need it, and the day you need it is not this one.

One practical note that does apply at every scale: this is an extension with a real security history, and 2026 saw an advisory that set a hard version floor. Whatever version is running, it should be current. That is true of the extension whether you are storing 241 vectors or 241 million.

The Server That Sleeps

Which brings in the second note, about things sitting on the machine switched off.

The instinct is sound and the mechanism has existed in the standard service manager on Linux for years. Rather than keeping a service running, you have the system hold the network port open on its behalf. Nothing is running and nothing is consuming memory. When a connection arrives, the manager accepts it, starts the real service, and hands the connection over. The caller experiences a slow first request and normal speed thereafter. After a period of idleness, the service shuts down again and the port goes back to being held by the manager.

For anything experimental this is close to ideal. You can have 6 half-finished things installed, occupying disk and nothing else, each of which comes alive when poked. The cost is the cold start, which for a small service is a second or 2 and for anything that loads a model into memory can be considerably longer.

And that cold start is exactly why this pattern fits the embedding problem so well, because of when embeddings are actually computed. A note arrives, gets embedded once, and the vector is stored forever. Notes do not arrive continuously. They arrive a few times a day, in bursts, often several at once from a single walk or drive. A service that wakes up, embeds 4 notes, and goes back to sleep is a perfect fit for a machine where memory is the scarce resource.

The counter-case deserves saying: if the embedding model is also serving interactive search queries, where a human is waiting, a multi-second cold start on the first search of the day is annoying. The resolution is not clever. Write-time embedding runs cold and nobody notices. If interactive search ever becomes a daily habit, keep it warm and pay the memory. Until then, do not pay for a habit you do not have.

What To Actually Build

So the shape of the project, one afternoon's research later, is smaller and better than the notes imagined.

Choose a multilingual embedding model, because the corpus is bilingual and this is the decision that determines whether anything works. Add a vector column. Do not create an index, and specifically resist creating one, because at this size an index makes queries slower rather than faster and adds a maintenance obligation for no benefit. Embed each item once, at write time, in a service that sleeps between bursts.

Then build the near-duplicate check first, because it is the feature that actually earns something. Every new note gets compared against everything, and when the similarity is high, a line is appended saying what it echoes. No interface. No dashboard. No new surface to maintain.

Search comes second, and it is a fusion of the vector results with the keyword index that is already in place, combined by rank rather than by score.

And there is a last thought that the arithmetic keeps pointing at. An inbox of 241 open items is not suffering from a retrieval problem. Nothing in it is hard to find. What it is suffering from is that things went in and never came out, and the reason they never came out is not that they could not be located. Better search on a pile that nobody revisits produces a better-indexed pile.

The near-duplicate detector is interesting precisely because it is the one feature that pushes back. It does not help you find things. It tells you that you have already had this thought, twice, and did nothing about it either time. That is a considerably less comfortable feature than semantic search, and considerably more likely to change anything.