What an Embedding Actually Is
Run the sentence “the pump is leaking” through an embedding model. What comes back is a list of numbers — 384 of them, or 1,024, or 3,072, depending on which model you picked. That list is the embedding. It is not a summary, not a compressed copy of the sentence, and no part of it is readable. Position 47 does not mean “mechanical” and position 300 does not mean “urgent”. Nobody, including the people who trained the model, can tell you what any single number is for.
The vector is good for exactly one thing: comparison. Ask how close it sits to the vector for “hydraulic unit is losing oil” and you get a number back — usually cosine similarity, between −1 and 1. That number is the entire product. Every embedding-based system you have ever used is built on top of it and nothing else.
Which leads straight to the rule people learn the expensive way: a vector is only comparable to vectors from the same model. Coordinates from OpenAI's text-embedding-3-large and coordinates from a self-hosted multilingual-e5-large describe two unrelated spaces that happen to have a similar shape. Comparing across them does not crash. It returns a plausible-looking number that means nothing. Hold on to that — it comes back later as the most expensive trap in this article.
Why Similar Meaning Lands Nearby
Nobody designed that space. It is residue from training. The model was shown enormous numbers of text pairs that belong together — a question and its answer, a headline and its article, a German sentence and its English translation — and was pushed to place each pair close while shoving randomly drawn pairs apart. Do that for long enough over enough of the internet and text that tends to occur in the same contexts ends up in the same neighbourhood. Geometry falls out of statistics.
That mechanism is worth stating precisely, because every misunderstanding downstream comes from being vague about it. The model did not learn meaning. It learned which texts tend to be about the same thing. “Similar” means “these two would plausibly turn up in the same conversation” — not “these say the same thing”, and certainly not “these are both true”.
The consequence is uncomfortable. “Water damage is covered under this policy” and “water damage is not covered under this policy” are about the same thing: same topic, same vocabulary, one word apart. An embedding model puts them almost on top of each other. For a regional insurer building search across its policy wordings, that is not an edge case — that is the entire job. Exclusions, exceptions, and safety instructions are precisely the passages where the distinction decides the answer, and precisely the passages where similarity is blind to it.
What That Buys You: Retrieval and Classification
Retrieval is the familiar half. Split documents into chunks, embed each chunk, store the vectors, and at query time embed the question and fetch its nearest neighbours. That is the front end of every RAG system — the full pipeline has considerably more in it, and the database underneath is where the interesting engineering about approximate search lives. Embeddings are one stage of that pipeline, and, usefully, rarely the stage that is broken.
The second half gets ignored, and it is the one I push most companies toward first. Embeddings make an excellent classifier and almost nobody uses them as one. If you have a thousand emails a human already sorted into “spare part enquiry”, “complaint”, “invoice question” and “other” — and every company has exactly that, sitting in a shared mailbox — you can embed all of them, fit a logistic regression on top in about ten lines of scikit-learn, and have a router that answers in milliseconds, costs nothing per call, runs on a CPU on your own hardware, and produces a confusion matrix you can put in front of the person who owns the process.
The reflex now is to prompt an LLM instead. Sometimes that is right — when the categories are subtle, or you have no labelled examples, or the label needs a justification written next to it. But the LLM costs money per call, takes a second per call, quietly changes behaviour when the vendor ships a new version, and gives you an output you can only evaluate by reading it. The embedding classifier is testable the way ordinary software is testable. For high-volume, low-nuance routing, it is simply the better engineering, and it is an afternoon of work.
Choosing an Embedding Model
The honest answer: this matters less than the leaderboards make it look, and you cannot settle it from a leaderboard anyway. MTEB is a real benchmark and a reasonable way to cut the field down to a shortlist of three. It cannot tell you which of those three is best on your maintenance reports, because your maintenance reports were not in it. In practice the gap between the top of the list and the sensible middle is smaller than the gap that chunking decisions open up.
So do the boring thing instead. Take 50 real questions — out of the ticket system, not out of your imagination — write down which document should come back for each one, run your shortlist, and count how often the right document appears in the top five. That is an afternoon of work. It ends the argument with evidence, and it leaves you with an evaluation set you will still be using in two years, every time someone proposes a change.
On German, one warning and one non-warning. The warning: do not assume a model near the top of an English leaderboard reads German competently. Tokenisers trained mostly on English shred German compounds — Betriebshaftpflichtversicherung becomes a handful of fragments that individually mean nothing, and the precise term your user typed is the one the model handles worst. The non-warning: a strong multilingual model — multilingual-e5, bge-m3, jina-embeddings-v3, Cohere's or OpenAI's multilingual offerings — usually beats a small German-only model, because at this layer scale tends to beat specialisation. Use MTEB's German tasks to build the shortlist, then run your own 50 questions on it.
One more thing worth knowing if data residency is what is holding a project up: the embedding layer is the cheap part of going on-premise. Models like multilingual-e5-large or bge-m3 fit comfortably on a single mid-range GPU and will run on CPU for overnight batch indexing. It is the generation model that makes self-hosting expensive, not this.
Three Traps That Never Throw an Error
Each of these produces a system that runs, returns results, and is wrong. Silence is what makes them expensive.
Your vocabulary was not in the training data
An embedding model knows what a bolt is because the internet is full of bolts. It has no idea that at your company “A-Teil” means a part on the critical path, or that DIN 912 and DIN 913 are different fasteners rather than two spellings of the same string. Part numbers, article codes, internal abbreviations, project shorthand: the tokeniser chops them into fragments and the fragments look alike. A machine builder's technician searching for a spare part by its number is the single worst case for pure vector search.
The fix is not a better embedding model. No model has your parts list. The fix is to stop asking embeddings to do a job they cannot do — run BM25 keyword search next to the vector search and merge the results, and keep the part number in a metadata field you can filter on exactly. For any corpus with identifiers in it, hybrid search is not a refinement you add later. It is the baseline you start from.
One chunk, one point
Every chunk gets exactly one vector, however much text is in it. A tight paragraph about warranty periods lands squarely in the warranty neighbourhood. A five-page chunk covering warranty, delivery terms and liability lands in the average of the three — which is a location in the space where nothing lives. It is no longer near “warranty”. It is near nothing in particular, and it will lose to a mediocre small chunk that is at least about the right topic.
That is the mechanism behind advice you have probably already heard: chunk small, chunk along structure. It is also why “let's use a model with an 8k context window, then we can embed whole documents” is a trap dressed as an upgrade. The constraint was never how much text fits through the model. The constraint is that one vector cannot point in three directions at once.
Two kinds of stale, and only one is obvious
The obvious kind: the document changed and the index did not. The price list is from March, the answer confidently quotes March, the customer gets a March price. Every RAG project plans for this — reindex on change, timestamp everything, expire what you cannot verify. It is a solved problem the moment somebody owns it.
The kind that catches teams is the other one: the model changed. You upgrade to a newer embedding model, or your provider deprecates the version you indexed with, or an engineer swaps in an e5 model without noticing that the e5 family expects queries prefixed with query: and documents with passage: — a convention that lives in one line of the model card and nowhere else. Nothing errors. Cosine similarity will happily compare two vectors from two unrelated spaces and hand back a perfectly reasonable-looking number. Quality does not collapse to zero, which would at least be visible. It degrades to mediocre, and nobody notices, because nobody kept a baseline.
So: write the model name, its version, and the prompt convention into the index metadata, and make the query path assert the match before it searches. Treat re-embedding as a migration with a cutover, not a config change. And notice the quiet argument this makes for self-hosting the embedding model — a vendor can retire a model on their schedule, and when they do, your index gets rebuilt on their schedule too.
Where Tippel Fits
Almost every embedding problem I get called into turns out to be one of those three, and none of them is fixed by a better model. That is the practical takeaway worth carrying: the embedding layer is rarely the bottleneck, and the effort belongs in chunking, in hybrid retrieval, and in having 50 real questions you can measure against.
If you want to know whether a retrieval or classification idea holds up on your actual documents, that is what the AI Readiness Check is for: one use case, your data, a measured result, fixed price, and an honest go/no-go at the end. Or get in touch and describe the corpus — it usually takes one call to tell whether the hard part is the model or the documents.