RAG calculator: how much does document vectorization cost

What is a chunk and why documents are split into parts

When you build an AI assistant over documents, the model cannot "read" an entire archive at once. That is why each document is first split into smaller units called chunks. A chunk is usually a few paragraphs, a single section, or part of a page that makes sense on its own.

The logic is simple: when a user asks a question, the system does not search the whole document but finds the most similar chunks and sends only those to the AI model as context. Smaller chunks mean more precise retrieval, but also more vectors to calculate and store. Larger chunks reduce the number of vectors but can "dilute" the information, so the model gets half a page of text in which only one sentence is relevant.

A good RAG calculator therefore takes three inputs: the number of documents, the average chunk size, and the price of the embedding model. From that it calculates the total number of chunks, the cost of vectorization, and the space those vectors take up in the database. This is useful primarily for planning, because many people underestimate how many chunks a few thousand documents produce.

How to calculate the number of chunks and embedding cost

In practice, the procedure is as follows:

1. Estimate the total number of tokens in the corpus. A rough guideline: one page of text has about 500 to 700 words, which is approximately 700 to 1000 tokens.
2. Choose a chunk size, for example 500 tokens.
3. Add overlap, for example 15 percent, so the effective step size is about 425 tokens.
4. Divide the total number of tokens by the step size to get the number of chunks.
5. Multiply the number of chunks by the embedding model price per token.

A concrete example: 200 documents of 20 pages each gives about 4 million tokens. With a chunk of 500 tokens and 15 percent overlap, you get approximately 9400 chunks. If the model charges for vectorization per token, the cost is calculated on those 4 million tokens, not on the number of chunks, because overlap means part of the text is processed twice.

Space in the vector database depends on the vector dimension and the data type. A vector with 1536 dimensions in a 4 byte format takes up about 6 KB, so 9400 chunks require about 56 MB for the vectors alone, plus metadata and indexes.

Which overlap and chunk size to choose

Overlap exists so that meaning is not lost at the boundary between two chunks. If a sentence is cut in half, the model that receives only the second part may not understand what it is about. That is why the last 10 to 20 percent of the text is repeated at the start of the next chunk.

Practical tips:

• For general documentation and blogs, start with 400 to 600 tokens with 10 to 15 percent overlap.
• For technical instructions and frequent lists, use smaller chunks, because users are looking for a specific step.
• For contracts and legal regulations, split by structure, for example by article and paragraph, not by token count. A chunk that contains a whole article makes much more sense than a cut at an arbitrary point.
• Always save metadata: document name, page number, and section title. Without that, the AI assistant's answer has no source the user can verify.

Expert tip: you do not choose the chunk size once and for all. Run a small test with 20 to 30 real questions and compare whether the system finds the right chunk. If it often misses, the problem is usually in the text splitting, not in the model.

Most common mistakes when planning vectorization

The first mistake is calculating cost per document instead of per token. The embedding price depends on the amount of text, not on the number of files, so a PDF with scanned images and a PDF with clean text do not have the same cost.

The second mistake is neglecting re vectorization. When you change the embedding model or the splitting method, everything has to be processed again. So plan versioning in advance and the ability to replace old vectors with new ones without interrupting operations.

The third mistake is focusing only on the cost of vectorization. It is usually a small part of the overall work, while most of the time goes into cleaning documents, removing headers and footnotes, and checking retrieval quality.

If the corpus grows to tens of thousands of documents or you have multiple sources that change often, planning the pipeline for processing and updating becomes more demanding than the calculation itself. In such situations, EVI Solutions can help with the architecture of the RAG system and the automation of data preparation.