AI & Machine Learning for Developers
Vector Databases
A vector database is a system built to store large numbers of embedding vectors and quickly find the ones most similar to a given query vector. Regular databases are built to look things up by an exact key or a range (an ID, a date); a vector database is built for a different question — "which of these million vectors are closest to this one?" — which needs different indexing techniques to answer fast enough to be usable.
Why it matters
- Exact search doesn't scale to "find similar" at any real size
- Comparing a query vector to every stored vector one by one (brute force) is accurate but gets too slow once a collection reaches a large enough number of vectors, which is the problem vector databases exist to solve.
- It's the storage layer underneath most RAG systems
- Retrieval-augmented generation, covered in this section's dedicated guide, needs somewhere to store and search document embeddings quickly, and a vector database is the usual answer.
- The indexing method involves a real trade-off
- Fast approximate search methods return results that are very likely, but not guaranteed, to be the true closest matches — trading a small amount of accuracy for a large amount of speed, and that trade-off is a genuine engineering decision, not a bug.
- It's a distinct component from the embedding model itself
- The embedding model decides what the vectors mean; the vector database decides how they're stored and searched — conflating the two makes it harder to reason about where a retrieval problem actually lives.
What makes vector search different
A traditional database index (like a B-tree behind a SQL WHERE clause) is built around exact matches and ordered ranges — great for "find the row where id = 42" but not built to answer "find the rows most similar to this one" in a high-dimensional space. Vector databases instead use indexing structures purpose-built for approximate nearest-neighbor search — finding vectors that are close, not identical, to a query vector, across possibly hundreds of dimensions. Alongside the vector itself, most systems let you store metadata (a source document ID, a timestamp, a category) and filter on it, so a query can mean "find vectors similar to this one, but only among documents tagged as published this year."
Approximate search and the speed/accuracy trade-off
Checking a query vector against every single stored vector (an exhaustive or "brute force" search) always finds the true closest matches, but the amount of work grows with the size of the collection, which becomes impractical at scale. Approximate nearest-neighbor algorithms trade a small, usually acceptable, chance of missing the single best match in exchange for search times that stay fast as the collection grows. This is a genuine, visible trade-off: an application that needs guaranteed exact results (rare in practice for this kind of similarity search) has to weigh that against one that tolerates "almost certainly the best match, found quickly." Most real-world systems accept the approximate version because the difference is rarely noticeable in practice, but it is worth knowing the guarantee has been traded away.
Mistakes people make here
- Treating a vector database as a full replacement for a regular database
- Most systems still need exact lookups, transactions, and structured queries that vector search isn't built for; vector databases typically sit alongside a regular database, storing embeddings and pointers back to the source records, not replacing the whole data layer.
- Forgetting to re-embed and re-index after source content changes
- A vector database stores whatever vector it was given; if the underlying document changes but its embedding isn't regenerated and re-stored, search results will quietly point at stale content.
- Not filtering by metadata when it's available
- Pure vector similarity can surface results that are semantically close but contextually wrong (the right topic, the wrong customer's data, for instance); combining similarity search with metadata filters is usually necessary, not optional.
- Assuming higher similarity always means a better result for the user
- A vector being the mathematically closest match doesn't guarantee it's the most useful or correct answer to show — similarity ranking is an input to a good result, not a guarantee of one.
Strengths and trade-offs
Where it is strong
- Purpose-built indexing makes similarity search over large collections practical in a way exhaustive comparison isn't.
- Combining vector similarity with metadata filtering supports queries that are both semantic ("similar meaning") and structured ("only from this category") at once.
- Most vector databases are designed to scale horizontally, so growing the collection is a capacity question rather than a redesign.
The trade-offs
- Approximate search trades a small, usually acceptable chance of a missed best-match for much faster queries at scale.
- It adds an entirely new piece of infrastructure to operate, secure, and keep in sync with the source data it was built from.
- Search quality is bounded by the embedding model's quality, not just the database's — a good vector database can't fix a poor embedding.
Who needs this
Developers building semantic search or a retrieval-augmented generation system need to understand what a vector database does and doesn't guarantee. If your application never needs to find "similar" items and only ever looks things up by exact key, this is not something you need day to day.
Questions about vector databases
- Is a vector database a specific product?
- No, it's a category. There are dedicated vector database products, and there are extensions that add vector search to an existing general-purpose database; both approaches exist and the right choice depends on the rest of the system.
- Do I need one for a small project?
- Not necessarily. For a small collection, comparing a query vector against every stored vector directly can be fast enough and much simpler to operate than adding a dedicated system.
- Does a vector database understand the meaning of what it stores?
- No. It stores and searches whatever vectors the embedding model produced; the meaning, and any errors in capturing that meaning, come entirely from the embedding step, covered in this section's embeddings guide.
- Is approximate search a problem in practice?
- Usually not — well-tuned approximate methods return results that are very close to exhaustive search nearly all the time. It becomes worth scrutinizing specifically in applications where missing the single best match has real consequences.