Chunk Size Calculator

Recommend chunk size and overlap for RAG documents.

FreeNo accountOn your device

Runs in your browser. How to verify

Loading tool…

About this tool

Enter how long the document is in tokens (or paste it), your target chunk size and the overlap, and get the number of chunks you will produce. It also shows the step between chunk starts. It calculates; it does not pick the best chunk size for you.

How to use

1
Step 1
Enter the document tokens, or paste the text.
2
Step 2
Set the target chunk size and overlap.
3
Step 3
Press Calculate.

How it works

Each chunk starts chunk − overlap tokens after the previous one. The chunk count is 1 + ceil((N − chunk) / step), or 1 if the document fits in one chunk. Characters per chunk are estimated at 4 per token. Chunk size is kept at 16 tokens or more, and overlap is kept below the chunk size.

step = chunk − overlap;  chunks = 1 + ⌈(N − chunk) / step⌉

Worked example

A 5,000-token document

Input
Document 5,000 tokens; chunk 512; overlap 64
Output
Chunk 512 tokens; step 448; estimated chunks 12; about 2,048 characters per chunk
Walkthrough
1 + ceil((5000 − 512) / 448) = 1 + ceil(10.02) = 12 chunks.

Limitations

  • The recommended size equals the target you enter; no optimisation is done.
  • It assumes a fixed-size sliding window. Sentence-aware splitters produce different counts.
  • The 4 characters per token figure is a rough English average.

Questions

How many chunks will my document make?
Enter the token count, chunk size and overlap. The estimate appears in the result.
How much overlap should I use?
10 to 20 percent of the chunk size is a common starting point.
What chunk size should I choose?
It depends on your embedding model and questions. 256 to 1,024 tokens is typical; test retrieval quality.
Can I paste the document instead of counting tokens?
Yes, the optional text field counts tokens for you.

Features

  • Chunk count
  • Step and characters per chunk
  • Counts pasted text

Related resources

Guides and videos from around the web—opens on YouTube or the publisher’s site.

  • Mean, Variance and Standard Deviation

    StatQuest with Josh Starmer

    Stats foundations behind many ML metrics tools.

    Watch on YouTube
  • TensorFlow 2.0 Complete Course

    freeCodeCamp.org

    Practical ML stack context next to AI utilities.

    Watch on YouTube