What a Vector Database Actually Does
Strip away the category name and one job remains: given a list of numbers, find the other lists of numbers that point in roughly the same direction. Those lists are embeddings — a chunk of your text turned into a few hundred or few thousand coordinates, positioned so that passages about the same thing land near each other. A vector database stores them and answers that nearest-neighbour question fast.
The word doing the quiet work here is approximate. Comparing your query against every vector in the corpus is exact and slow. Every production system instead uses an approximate nearest-neighbour index — HNSW, which builds a navigable graph through the vectors, or IVFFlat, which sorts them into buckets and only searches the closest few. Both trade correctness for speed: they answer in milliseconds and quietly miss some genuinely good matches. You tune that trade-off with a parameter (in HNSW, how many candidates to keep in play during the search).
That property is the one people forget. Your search index is lossy, and it never tells you what it lost. There is no error when the one chunk that answers the question was ranked 51st and your pipeline took the top 50. The model receives a plausible context and writes a fluent answer from it. Vector search does not break loudly; it degrades politely.
The Honest Answer: Probably pgvector
pgvector is an extension for PostgreSQL. It gives you a vector column, the usual distance functions, and HNSW indexing — inside the database that already holds the row the chunk came from, with the transactions, backups, roles and monitoring your team already understands. For most corpora that is not a compromise. It is the correct engineering answer.
Do the arithmetic before you go shopping, because vendor sizing pages are written for a scale that is not yours. A 1024-dimension vector in float32 is about 4 KB, so a million chunks is roughly 4 GB of raw vectors plus graph overhead. And a million chunks is a lot of company: at perhaps twenty chunks per document, that is around fifty thousand documents — every quote, manual, policy and contract a mid-sized business has produced in a decade. It fits in RAM on one ordinary server.
When a vendor says "scale," they mean billions of vectors and thousands of queries per second, because that is the problem that justifies a distributed system. You have millions of vectors and a few queries per minute from staff who are also doing other things. Only one of those is your problem. I would not stand up a self-hosted Milvus cluster for a 200,000-chunk corpus: you would be buying a distributed system's failure modes — sharding, rebalancing, node ops — to solve something a single Postgres box does not even find interesting.
The Criteria That Actually Decide
Three things override that default. Notice that only one of them is about size — and it is not the one the marketing pages lead with.
Scale, but the number that actually hurts
pgvector gets uncomfortable in the tens of millions of vectors, and the pain shows up in index build time and memory during rebuilds long before it shows up in query latency. Under roughly five million chunks with modest concurrency, stop shopping; you are optimising a problem you do not have. At fifty million and rising — a large document archive, a product catalogue across many markets — a purpose-built store earns its keep, and that is a real engineering decision rather than a preference.
Filtering, where vector search quietly breaks
This is the criterion that actually decides most real selections, and almost nobody raises it in the first meeting.
Take a regional insurer building an assistant over policy wordings and claims guidelines. No query is ever "search everything." Every query is "search the documents this user is allowed to see, in this line of business, valid on today's date." So the pipeline needs a metadata filter alongside the vector search — and where that filter runs decides whether the system works.
If you retrieve the top 50 by similarity and then filter in your application code, a filter that admits 2% of the corpus leaves you with roughly one usable chunk out of fifty. The model answers from that one mediocre passage, confidently. The system looks like it is working. This is the single most common way a demo that impressed everyone turns into a production assistant that gives thin, subtly wrong answers.
The fix is filtering that happens inside the index traversal, so the search only ever walks permitted vectors. Purpose-built stores such as Qdrant and Weaviate treat filtered search as a first-class feature; Elasticsearch and OpenSearch do it as an extension of the filtering they have always done. pgvector can do it too, but you have to know how the query planner behaves, and whether it is choosing your index or falling back to a scan. If your access model is a hard requirement — and for an insurer it is not negotiable — filter behaviour is the selection criterion, ahead of throughput and well ahead of brand.
Hybrid search, because your users type part numbers
Semantic search is genuinely bad at exact tokens. "DIN 933", "4-2291-B", "§ 34c" — an embedding maps these to a vague neighbourhood of similar-looking strings, because there is no meaning in a part number to capture. An automotive supplier whose engineers search by part number will find pure vector search infuriating, and they will be right.
The answer is hybrid: run keyword search (BM25) and vector search together and fuse the rankings. This is a genuine argument in Postgres's favour, because it has had full-text search for years — you can do both in one query against one system. It is also an argument for Elasticsearch or OpenSearch on text-heavy corpora: a mature keyword engine that added vector search is often a better shape for this problem than a vector engine that added keywords. Which retrieval mix you need is a pipeline design question, and it should be settled before anyone signs anything.
The Part Nobody Budgets: Re-Indexing
A vector index is derived data. It is not a system of record, and treating it as one is the mistake that turns a small decision into an expensive one.
Here is why it matters more than the benchmark table. Vectors produced by one embedding model are meaningless in another model's space. There is no conversion, no migration path, no clever trick: change the model and you re-embed every chunk. And you will change it — a better or cheaper model ships, or, far more often, you change your chunking strategy after seeing how retrieval behaves on your actual documents. Re-indexing is not an edge case. It is a Tuesday.
Why This Is Rarely the Interesting Decision
Retrieval quality is decided upstream of the database, every time. What got ingested, how it was chunked, whether metadata survived the import, whether a reranker sits between retrieval and the prompt — those determine whether the right passage is findable at all. The database only executes the search. It has no opinion about whether the answer is in there.
I have never seen a project fail because someone picked the wrong vector database. I have seen several struggle because the chunks were wrong, because access control was bolted on after the fact, or because nobody could rebuild the index without losing a weekend. The store is also one of the more reversible decisions in the stack: chunks, embeddings and metadata are portable, and swapping it is usually days of work, not a rewrite. Reversible decisions deserve proportionate deliberation — which is to say, not much.
So: pick the boring option that handles filters and hybrid search, that your team can back up and restore, and that you can rebuild from source on demand. Then go and argue about chunking, which is where the answers actually come from.
Where Tippel Fits
Most of the vector database question dissolves once someone has looked at your corpus: how many chunks it really is, how selective the access filters have to be, and whether your users search by meaning or by part number. Those three facts pick the store in an afternoon. Our AI Readiness Check is a paid, timeboxed look at exactly that — your real documents, a working retrieval slice, and an honest verdict on what a production build would take. If you would rather see the broader shape of the work first, the services overview covers how we scope and build these systems, or just get in touch and describe the corpus.