What a vector database looks like from the inside
Under the hood, a vector database just turns meaning into points on a map, measures how far apart they sit, and takes a shortcut so it never has to measure them all. Scroll, and the page flies you through each piece until the product stops looking like magic.
My team built a vector database from the ground up and took it into the market against the leaders. I have also shipped AI products that lean on vector databases for everything they remember. Both jobs taught me the same thing: the usual explanation of these products, that they store embeddings and find similar ones, is true but tells you almost nothing.
So here is the explanation that actually helps. A vector database does three things: it turns meaning into coordinates, measures the distance between them, and cheats on the measuring so it can finish before your user gives up. Once you understand those three, it gets much easier to judge what a vendor tells you, what you are really paying for, and why the answers sometimes come back wrong.
Building the engine showed me what the mathematics costs. Building products on top of one showed me what buyers actually get for that money. This article is what I wish every buyer carried into a vendor conversation, laid out as five levels along one flight path. Each level has a working model you can grab, spin and click.
A vector database stores each item as a long list of numbers, called a vector or embedding, arranged so that similar meanings land close together. A search converts the query into the same kind of vector and returns the stored items nearest to it. Because comparing against every stored vector is too slow past a few hundred thousand items, the database uses an approximate index, usually HNSW or IVF, that looks at only a small part of the data and returns the nearest matches with high but not perfect accuracy.
Next stop · Level 1 · Meaning becomes a point on a map
Burn 1 of 6 · the spreadsheet planet
Meaning becomes a point on a map
An embedding is a scorecard. Imagine rating every object on a handful of properties: how alive it is, how big, how fast, how edible. Write the scores down in a fixed order and you have a vector. A cat might score [0.95, 0.25, 0.55, 0.05], a truck [0, 0.8, 0.7, 0]. There is nothing mysterious here. You have been looking at vectors in spreadsheets for years.
The trick is what happens when you treat the row as coordinates. Each property is an axis, so each object becomes a point, and objects with similar scores sit near each other. That is the entire idea. A vector database is a database of points.
On the left, sixteen objects sit in a spreadsheet with their scores underneath. Keep scrolling and watch the spreadsheet become a space: each object flies to the spot its scores describe. Three of the four scores become width, height and depth. The fourth, Edible, is still in the data; a screen just cannot draw it, and a real model juggles 1,536 of these at once.
A real embedding model does the same thing with one difference that matters. Nobody chooses the properties. The model learns them from text, and it learns a lot of them: OpenAI's text-embedding-3-small, a common choice, produces 1,536 numbers per input. You cannot name axis 731. You can only trust that items the model considers similar end up close.
The famous example came from the word2vec papers in 2013, where vector(king) − vector(man) + vector(woman) landed near vector(queen). Direction on the map carried meaning.
The embedding model sets the ceiling on everything that comes after it. The database can only ever find what is close, so if the model puts the wrong things close together, no index will save you. Pick the model first and the database second, and test it on your own documents, not on the vendor's demo set.
Next stop · Level 2 · Distance is the whole product
Burn 2 of 6 · the ruler planet
Distance is the whole product
If items are points, then similar means near, and you need a ruler. There are three rulers in common use, and they disagree.
Euclidean distance is the straight line between two points. Dot product measures how much two arrows point the same way, scaled up by their lengths. Cosine similarity is the dot product with the lengths divided out, so it measures the angle only.
cosine cos θ = (a · b) / (|a| |b|)
euclid d = √ Σ (aᵢ − bᵢ)²
Grab the arrowheads on the left. Watch cosine stay the same when you stretch an arrow and change only when you turn it.
Why does a founder need to care about rulers? Because the ruler has to match the one the embedding model was trained for, and a mismatch does not crash anything. Results look fine and are slightly wrong, forever. Most modern text embedding models are built for cosine or dot product, and the model's documentation says which. When a database proudly advertises support for all three, what it is really doing is handing that decision to you.
Scale every vector to length one before you store it, which engineers call normalising. Then cosine, dot product and Euclidean all rank results the same way, and the cheapest of the three, the dot product, is safe to use. Tick the box above to see all three agree.
Next stop · Level 3 · Checking every vector hits a wall
Burn 3 of 6 · the brute force planet
Checking every vector hits a wall you can calculate
The honest way to find the nearest neighbour is to measure the distance to every stored vector and keep the smallest. Engineers call this brute force, or a flat index. It is exact, simple, and it has a cost you can work out on a napkin.
One search against N vectors of d dimensions means N × d small multiplications, and it reads the whole table from memory. Ten million vectors of 1,536 dimensions is 15 billion multiplications and 61 GB of reading, for every single search. Move the sliders and watch the numbers.
Side quest: why "nearest" gets fragile in high dimensions
There is a second, stranger problem. In high dimensions, random points end up at almost the same distance from everything. The histogram below draws 1,500 random points and measures their distance from one query. At 2 dimensions the distances spread wide. At 1,024 they bunch into a narrow band. The gap between the nearest point and a typical point shrinks, so a small mistake by an index flips the answer. This is the curse of dimensionality, described by Beyer and colleagues in 1999, and it is one reason approximate search is both possible and risky.
Below roughly 100,000 vectors, brute force is usually fine and you may not need a vector database at all. That is my rule of thumb, not a law. A few lines of Python, or a plain scan inside the database you already run, answers in milliseconds at that size. The wall arrives when N climbs into the millions, and that is where the shortcut comes in.
Next stop · Level 4 · The shortcut is a map with motorways
Burn 4 of 6 · the HNSW planet
The shortcut is a map with motorways
HNSW stands for Hierarchical Navigable Small World graph, which is a mouthful for a simple idea. Connect every point to a few of its nearest neighbours, so the data becomes a road network. Then build a few thinner layers on top, with fewer points and longer roads, like motorways over country lanes.
A search starts on the top layer at a fixed entry point and keeps stepping to whichever neighbour is closest to the query. When no neighbour is closer, it drops down a layer and repeats. On the bottom layer it keeps a short list of the best candidates, whose length is a setting called ef, and it stops when the list stops improving.
On the left the layers are stacked as real floors. Click anywhere on the bottom floor to drop a query, then watch the walker start at the top, cross the long roads, drop a floor, and finish in the lanes. The counter shows how many distances it measured against the 400 it would take to be certain. The graph is a real HNSW index, built in your browser when the page loaded.
The word in "approximate nearest neighbour" that should worry you is approximate. The walk can get stuck in a dead end and return the second-best answer. Keep ef at 1, run ten searches, and you will see it happen. The counter keeps an honest recall score: the share of searches that returned the true nearest point. The 2016 paper by Malkov and Yashunin that introduced HNSW shows the trade directly: raise ef, or the number of roads per point, and recall climbs toward 100% while searches slow down and memory grows.
Approximate is a dial you own, and recall is a number you must measure on your own data. The published curves on ann-benchmarks.com are for public datasets. Your documents, your embedding model and your queries sit on a different curve. The test is cheap: take 1,000 real queries, work out the exact top 10 with brute force once, and count how many of them the index returns. That number, recall@10, belongs in your vendor contract. I use the same method whenever I evaluate an index at tens of millions of vectors, and I will publish the full procedure on this site.
Next stop · Level 5 · Memory is the bill
Burn 5 of 6 · the quantization planet
Memory is the bill, so the database shrinks the numbers
HNSW is fast but hungry. The graph and the vectors both live in RAM. At scale, memory is the line on the invoice, and there are two ways to cut it.
The first is to search less. IVF, the inverted file index, groups the points into cells with a standard clustering method called k-means, then searches only the few cells nearest the query, a count called nprobe. It is a default building block in Meta's FAISS library and the backbone of most billion-scale systems.
The floor on the left is 600 points cut into 16 cells. Click it to drop a query. Only the lit cells get searched. Raise nprobe and watch candidates and recall climb together.
The second way is to store less. Each number in a vector normally takes 4 bytes. Round it down to a single byte, which is called scalar quantization, and the vector shrinks to a quarter of its size. Product quantization goes further: it splits the vector into chunks and replaces each chunk with the nearest of 256 learned patterns, so a 1,536-dimension vector can fit in 96 bytes or fewer. The price is error. The stored vector is no longer exactly the one the model produced.
The skyline floating above the floor is one vector, the first 16 of its 48 numbers drawn as towers. Choose a precision and watch the red outlines, the values rebuilt from the compressed form, drift away from the solid towers. The page then runs 60 searches against its own test set of 400 vectors to count how often the top result survives.
| precision | bytes / vector | top-1 survives (60 queries) |
|---|---|---|
| float32 | 6,144 | – |
| int8 | 1,536 | – |
| 4-bit | 768 | – |
| 1-bit | 192 | – |
Two things to read off that table. int8 costs almost nothing, which is why it is the default in most products. And 1-bit falls apart here because keeping only the plus or minus sign of each number throws away too much of a short vector. Real systems use 1-bit codes only on vectors of 1,000 or more dimensions, where the sign pattern holds much more, and they re-check the top results at full precision.
Every production system I have worked on ended up combining these: an IVF or HNSW index over compressed vectors, with a final step that re-checks the top candidates against the original, uncompressed vectors. None of that is a vendor secret, just the arithmetic of the last three levels put to work.
Next stop · Final approach · The decision on your desk
Burn 6 of 6 · look back along the path
The decision on your desk
Look back along the flight path on the left and you can see all five levels at once. Here is how I turn them into choices when I am the one deciding. The thresholds are rules of thumb from my own work, so treat them as starting points.
| Scale | What usually works | What to watch |
|---|---|---|
| Under 100k vectors | Brute force (a flat index) in the database you already run, or NumPy | Nothing yet. Measure latency before adding a system. |
| 100k to 10M | HNSW in memory | RAM is vectors plus graph. Recall@10 on your own queries. |
| 10M to 1B and beyond | IVF with product quantization, re-rank the top candidates at full precision | nprobe tuning, recall lost to quantization, rebuild time when the data changes. |
| Any scale with metadata filters | An index that filters during search, or partition by tenant up front | Filters can collapse recall. Test filtered queries separately. |
Five questions to ask before you sign
- Which distance metric, and does it match my embedding model's card?
- What is recall@10 on my queries, at the latency you quote?
- How many bytes per vector do you store, and is any of it compressed?
- What happens to recall when I filter by metadata?
- How do inserts and deletes affect the index, and when does it rebuild?
A vendor who can answer all five in numbers is selling you an index. A vendor who answers in adjectives is selling you a brochure.
Five questions, no notes
Each correct answer is worth 20 XP. The explanations are the article in miniature, so a wrong answer still pays.
Three pieces of mathematics and one shortcut
Embeddings turn meaning into coordinates. A metric measures distance. An approximate index skips most of the measuring. Everything else a vector database does is packaging around those three.
The next step is small. Before you sign anything, take 1,000 real queries from your product, work out the exact answers once with brute force, and measure recall@10 against the index you are being sold. That number will tell you more than any demo.
If this was useful, the companion piece on how I design memory for AI agents, with hot and cold paths over a multi-index store, goes one level deeper and is next on this site. I publish data-backed articles here and post the short versions on LinkedIn and X.
A vendor who answers in numbers is selling you an index. A vendor who answers in adjectives is selling you a brochure.
- Malkov, Y. and Yashunin, D. (2016, revised 2018). Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs. arXiv:1603.09320.
- Johnson, J., Douze, M. and Jégou, H. (2017). Billion-scale similarity search with GPUs. arXiv:1702.08734. FAISS documentation: github.com/facebookresearch/faiss/wiki.
- Jégou, H., Douze, M. and Schmid, C. (2011). Product Quantization for Nearest Neighbor Search. IEEE TPAMI. ieeexplore.ieee.org/document/5432202.
- Mikolov, T. et al. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv:1301.3781.
- Beyer, K., Goldstein, J., Ramakrishnan, R. and Shaft, U. (1999). When Is "Nearest Neighbor" Meaningful? ICDT 1999. doi:10.1007/3-540-49257-7_15.
- Aumüller, M., Bernhardsson, E. and Faithfull, A. ANN-Benchmarks. ann-benchmarks.com.
- OpenAI embeddings guide (model dimensions). platform.openai.com/docs/guides/embeddings.