Chunk Size Calculator
Recommend chunk size and overlap for RAG documents.
About this tool
Enter how long the document is in tokens (or paste it), your target chunk size and the overlap, and get the number of chunks you will produce. It also shows the step between chunk starts. It calculates; it does not pick the best chunk size for you.
How to use
How it works
Each chunk starts chunk − overlap tokens after the previous one. The chunk count is 1 + ceil((N − chunk) / step), or 1 if the document fits in one chunk. Characters per chunk are estimated at 4 per token. Chunk size is kept at 16 tokens or more, and overlap is kept below the chunk size.
step = chunk − overlap; chunks = 1 + ⌈(N − chunk) / step⌉
Worked example
A 5,000-token document
- Input
- Document 5,000 tokens; chunk 512; overlap 64
- Output
- Chunk 512 tokens; step 448; estimated chunks 12; about 2,048 characters per chunk
- Walkthrough
- 1 + ceil((5000 − 512) / 448) = 1 + ceil(10.02) = 12 chunks.
Limitations
- The recommended size equals the target you enter; no optimisation is done.
- It assumes a fixed-size sliding window. Sentence-aware splitters produce different counts.
- The 4 characters per token figure is a rough English average.
Questions
How many chunks will my document make?
How much overlap should I use?
What chunk size should I choose?
Can I paste the document instead of counting tokens?
Features
- Chunk count
- Step and characters per chunk
- Counts pasted text
Related resources
Guides and videos from around the web—opens on YouTube or the publisher’s site.
Mean, Variance and Standard Deviation
StatQuest with Josh Starmer
Stats foundations behind many ML metrics tools.
Watch on YouTubeTensorFlow 2.0 Complete Course
freeCodeCamp.org
Practical ML stack context next to AI utilities.
Watch on YouTube
- Metrics and scoring
scikit-learn
Precision, recall, F1 definitions used by AI metric tools.
- OpenAI Research
OpenAI
Research notes for people exploring AI tooling trends.