Does changing the chunk size help?
There is no universal best chunk size. Each configuration is measured for retrieval quality and for what it costs, so the trade-off is visible instead of assumed.
Executive summary
Which chunk type and size work best, how fast each is, and why. Every number comes from the benchmark below.
Structure-aware chunks of about 256 tokens, searched by words and meaning together.
Cut the filing along its own sections and sentences, not into equal token windows. This found as many answers as any setup, found the most when the amount of text is held fixed, and hands the AI about 2.7× less text per answer than the best fixed-size setup. It is the configuration the Ask page uses.
| Setup | Top 5 | Top 10 | Within 2,000 tokens | Text read | Passages stored | Search time | Build time |
|---|---|---|---|---|---|---|---|
| Fixed-size 128 | 33% | 46% | 50% | 640 | 30,871 | 58 ms | 5.9 min |
| Fixed-size 256 | 31% | 48% | 42% | 1,280 | 15,461 | 39 ms | 7.7 min |
| Fixed-size 384 | 42% | 60% | 42% | 1,919 | 10,327 | 28 ms | 9.1 min |
| Fixed-size 480 | 48% | 60% | 44% | 2,400 | 8,269 | 23 ms | 10.0 min |
| Structure-aware 128 | 48% | 50% | 58% | 458 | 41,408 | 61 ms | 5.6 min |
| Structure-aware 256Recommended | 48% | 60% | 60% | 895 | 22,161 | 40 ms | 7.0 min |
| Structure-aware 384 | 46% | 65% | 58% | 1,312 | 16,163 | 34 ms | 7.9 min |
| Structure-aware 480 | 42% | 63% | 50% | 1,498 | 13,884 | 29 ms | 9.8 min |
Type: follow the document's structure
Fixed-size chunks cut every N tokens wherever that lands: 96% of their cuts fall in the middle of a sentence, and at 128 tokens 16% of answers were split across two passages, so no single result held the whole answer.
Structure-aware chunks break at sections, paragraphs and sentences (mid-sentence cuts: 1%) and kept every answer whole. A passage that is one coherent idea makes a sharper match, for meaning and for keywords alike. The gap is widest at small sizes: 48% against 33% at 128 tokens, 48% against 31% at 256.
Size: about 256 tokens
Too small (128). It often puts the exact sentence first, but a short passage loses the words around it (the company, the section heading), so when it is not first it rarely ranks at all: 50% within ten results, against 60% at 256. It also stores the most passages (41,408).
Too large (384–480). One passage then covers several topics and its vector becomes an average of them. Searching by meaning alone found 44% at 256 tokens but 29% at 384 and 33% at 480, and every answer costs more text to read.
Speed: three different speeds
Building the index. Smaller chunks embed faster, because the model's work grows faster than passage length: 5.6 min for 100 filings at 128 tokens, 9.8 min at 480.
Searching. Every setup searched in tens of milliseconds. On the live site a whole search takes under a second, mostly keyword matching and spelling correction, so chunk size barely matters here.
Writing the answer. This is the one that counts: the AI's time and cost grow with the text it reads. 256-token structure-aware chunks read 895 tokens per answer; the best fixed-size setup reads 2,400 for the same answer rate.
Words, meaning, or both
At the recommended setting, keywords alone found 40%, meaning alone 44%, and both together 48%. Keywords catch exact names and figures (“Lennar”, “22.3%”); meaning catches paraphrase (“margins fell” for “gross margins decreased”). Combining them is not a guarantee: for fixed-size 480, keywords alone scored 52%, more than both together (48%).
Finding the right company's filing was never the problem: it was in the top five for every question in every setup. The misses were all about choosing the right passage inside it, which is what chunking controls.
Answers that span several sentences
Measured with a second set of questions whose answers need two to four consecutive sentences, scored on whether the whole answer reached the AI. The full table is in the technical summary on the How it works page.
Structure still wins. Structure-aware 256 got the whole answer into the top five 54% of the time, reading 828 tokens. The best fixed-size setup, fixed-size 480, managed 42% while reading 2,400. So the recommended setting holds for longer answers too.
Find small, read big helps blind cuts, not good ones. Adding each result's neighbouring passages raised fixed-size chunks by 12 points on average but structure-aware ones by only 3, and roughly tripled the text read (best with neighbours: 58% with structure-aware 384, at 3,287 tokens). Use it only when answers routinely run past a paragraph. Stay at or under 480 tokens per chunk: the embedding model reads at most 512.
48 questions, 100 filings. With this many questions one question moves a score by about 2 points, so gaps of a few points (46% against 48%) are noise. The clear findings are structure-aware over fixed-size at small sizes, and that larger chunks cost much more text for little or no gain.
The details
The same real filings, cut eight different ways, asked the same questions with known answers.
- Yes. The same questions were answered from the top five results 31% of the time with fixed-size 256-token chunks and 48% with structure-aware 128-token chunks.
- 3 setups tied at 48%, but structure-aware 128-token chunks got there reading 458 tokens, against 2,400 for fixed-size 480-token chunks.
- Bigger is not free: 480-token chunks hand the model about 3× as much text to read (1,498 vs 458 tokens for five results), and the match gets diluted.
- Cutting along the filing's own sections and sentences beat blind token windows by 7 points on average at the same size.
- With the reading budget fixed at 2,000 tokens, structure-aware 256-token chunks found the most answers (60%).
- Finding the right company's filing was never the problem: it was in the top five for every question in every setup. The misses were about picking the right passage inside it, which is exactly what chunking controls.
48 questions with known answers, searched across 100 real 10-K filings. A question counts as answered when the top five passages contain the sentence that answers it. Measured 12 Oct 2026, 02:19.
| Chunk size | Fixed-size: answer found | Structure-aware: answer found |
|---|---|---|
| 128 tokens | 33% reads 640 tokens · answer kept whole 84% | 48% reads 458 tokens · answer kept whole 100% |
| 256 tokens | 31% reads 1,280 tokens · answer kept whole 92% | 48% reads 895 tokens · answer kept whole 100% |
| 384 tokens | 42% reads 1,919 tokens · answer kept whole 96% | 46% reads 1,312 tokens · answer kept whole 100% |
| 480 tokens | 48% reads 2,400 tokens · answer kept whole 100% | 42% reads 1,498 tokens · answer kept whole 100% |
Words, meaning, or both?
Each configuration was also searched by keywords alone and by meaning (vectors) alone. Combining them is what the Ask page does.
| Configuration | Words only | Meaning only | Both |
|---|---|---|---|
| Fixed-size 128 | 31% | 23% | 33% |
| Fixed-size 256 | 31% | 25% | 31% |
| Fixed-size 384 | 35% | 42% | 42% |
| Fixed-size 480 | 52% | 46% | 48% |
| Structure-aware 128 | 33% | 38% | 48% |
| Structure-aware 256 | 40% | 44% | 48% |
| Structure-aware 384 | 31% | 29% | 46% |
| Structure-aware 480 | 33% | 33% | 42% |
Question by question
The position at which each configuration first found the answer (top 10; bold is top 5). F = fixed-size, S = structure-aware. Open a question to see the sentence that answers it.
| Question | F128 | F256 | F384 | F480 | S128 | S256 | S384 | S480 |
|---|---|---|---|---|---|---|---|---|
LENWhat was Lennar's gross margin on home sales in 2025, and how does it compare with 2024?“Gross margins on home sales were $5.7 billion, or 17.7%, in the year ended November 30, 2025, compared to $7.5 billion, or 22.3%, in the year ended November 30, 2024.” | — | — | 1 | 1 | 6 | 2 | 4 | 6 |
LENWhy did Lennar's homebuilding margins fall?“gross margins decreased primarily due to a lower revenue per square foot and higher land costs year over year” | — | — | — | 7 | — | — | — | — |
PHMHow did PulteGroup's gross margin from home sales change in 2025?“gross margin from home sales, which decreased to 26.3% in 2025 versus 28.9% in 2024” | 3 | 2 | 8 | 3 | — | 1 | 4 | 10 |
PHMWhat was PulteGroup's home sale gross margin in the fourth quarter of 2025?“from 27.5% in the first quarter of 2025 to 24.7% in the fourth quarter of 2025” | 3 | 2 | 2 | 2 | 1 | 3 | 2 | 3 |
AMPHow much did Ameriprise's total assets under management grow in 2025?“Total AUM increased $127.0 billion, or 11%, to $1.3 trillion as of December 31, 2025” | — | — | 3 | 5 | — | — | — | 9 |
MRSHHow much consulting revenue did Marsh report for 2025?“Consulting revenue in 2025 was $9.8 billion, an increase of 7%, or 5% on an underlying basis.” | 5 | 1 | 2 | 3 | 4 | 1 | 3 | 4 |
LMTBy how much did Lockheed Martin's product sales rise in 2025?“Product sales increased $3.4 billion, or 6%, in 2025 as compared to 2024.” | — | — | — | — | — | — | — | — |
LMTWhich Lockheed Martin business areas drove the growth in product sales?“higher product sales of approximately $1.7 billion at MFC, $1.0 billion at Aeronautics, and $475 million at Space” | — | — | — | — | — | — | — | — |
DLTRWhat was Dollar Tree's comparable store sales growth for the year?“Net sales increased 10.4% to $19,395.7 million due to a 5.3% comparable store net sales increase” | — | — | — | — | — | — | — | — |
ORLYWhat was O'Reilly Automotive's gross profit as a percentage of sales in 2025?“Gross profit for the year ended December 31, 2025, increased 7% to $9.17 billion (or 51.6% of sales)” | — | 4 | 9 | 8 | — | 4 | 7 | 8 |
ORLYWhy did O'Reilly's gross margin percentage improve?“was due to improved acquisition costs and distribution operating efficiencies” | — | — | — | — | — | — | — | — |
GOOGHow fast did Google Cloud revenue grow last year?“an increase in Google Cloud revenues of $15.5 billion, or 36%” | 2 | 7 | 7 | — | 2 | 2 | 9 | 8 |
GOOGWhat were Alphabet's total revenues and growth rate?“Revenues were $402.8 billion, an increase of 15% year over year” | — | — | — | — | — | — | — | — |
FOXWhy did Fox's operating expenses increase in fiscal 2025?“Operating expenses increased $1.4 billion or 16% for fiscal 2025, as compared to fiscal 2024, primarily due to the approximately $1 billion impact of higher sports programming rights amortization and production costs driven by higher NFL costs” | — | — | 8 | 9 | — | 8 | 4 | 2 |
CMSHow large an annual electric rate increase did Michigan regulators authorize for Consumers Energy in March 2025?“authorizing an annual rate increase of $176 million” | 3 | 6 | 5 | 9 | 1 | 6 | 5 | 2 |
FIXHow many employees did Comfort Systems USA have at the end of 2025?“we had approximately 22,700 employees as compared to approximately 18,300 employees as of December 31, 2024” | — | 2 | 3 | 3 | — | 2 | 2 | 3 |
AONHow many people does Aon employ, and in how many countries does it operate?“we employed approximately 60,000 employees and conducted our operations in more than 120 countries” | 9 | 4 | 3 | 3 | 2 | 3 | 3 | 3 |
MOSHow many employees does Mosaic have, and how many are hourly?“We had 13,249 employees as of December 31, 2025, consisting of approximately 9,914 salaried and 3,335 hourly employees.” | — | — | — | 1 | — | 6 | 5 | 6 |
RMDHow many of ResMed's employees work in research and development?“we had approximately 10,600 employees, of which approximately 4,240 were employed in cost of sales activities including areas such as warehousing and manufacturing, 1,990 in research and development and 4,370 in sales, marketing and administration” | — | — | 2 | 3 | — | 1 | 1 | 1 |
ABNBHow many employees does Airbnb have?“As of December 31, 2025, we had approximately 8,200 employees.” | 1 | 2 | 3 | 1 | — | — | — | — |
AKAMWhat share of Akamai's employees work outside the United States?“with approximately 65% of those employees located outside of the U.S.” | — | 1 | 1 | 1 | — | 2 | 3 | 2 |
DPZHow many Domino's employees support its supply chain operations?“approximately 3,200 employees supporting our U.S. and Canadian supply chain operations” | 9 | 6 | — | 1 | 1 | 10 | 9 | 8 |
JBLAcross how many locations and countries does Jabil's workforce operate?“employees across approximately 100 locations in 30 countries” | 9 | 9 | 2 | 2 | 5 | 6 | 5 | 4 |
HSYWhat share of Hershey's sales goes to McLane?“approximately 27% of our consolidated net sales were made to McLane Company, Inc.” | 2 | 6 | 2 | 1 | 1 | 2 | 2 | 2 |
MCKHow dependent is McKesson on its largest customer?“sales to our largest customer represented approximately 24% of our total consolidated revenues” | — | — | — | — | — | — | — | — |
BRHow much of Broadridge's revenue comes from its largest client?“our largest client accounted for approximately 7% of our consolidated revenues” | 9 | — | — | — | 2 | — | 10 | 9 |
FTVWhat percentage of Fortive's sales were made outside the United States?“sales outside the United States accounted for approximately 44% of our total sales for the year” | 7 | 6 | — | — | 1 | 7 | — | — |
SLBHow much of SLB's revenue comes from outside the US?“our non-US operations accounted for approximately 82% of our consolidated revenue in 2025” | — | — | — | — | — | — | — | — |
CARRWhat portion of Carrier's net sales came from parts and service?“new equipment comprised 72% and parts and service comprised 28% of our net sales” | 3 | 1 | 6 | 1 | 2 | 1 | 1 | 1 |
CBOEHow concentrated is Cboe's customer base?“our top ten customers accounted for approximately 57% of our revenues” | — | — | — | — | — | — | — | — |
JBHTWhat share of J.B. Hunt's revenue came from its ten largest customers?“our top 10 customers, based on revenue, accounted for approximately 33% of our revenue” | — | — | — | — | — | — | — | — |
MRVLHow much of Marvell's revenue comes from customers in Asia?“Sales shipped to customers with operations in Asia represented approximately 75% of our net revenue in fiscal 2025” | 1 | 2 | 7 | 5 | 1 | 1 | 4 | 2 |
MNSTWhich bottler is one of Monster Beverage's biggest customers?“Coca-Cola Consolidated, Inc. accounted for approximately 10% of our net sales” | — | 10 | 7 | — | — | — | — | — |
CFWho is CF Industries' largest customer?“CHS was our largest customer in 2025 and accounted for approximately 13% of our consolidated net sales.” | — | — | 7 | — | — | — | — | — |
MCHPWhat share of Microchip's sales goes through distributors?“Sales to distributors accounted for approximately 45% of our net sales in fiscal 2025” | — | — | — | — | — | — | — | — |
KHCHow much of Kraft Heinz's sales come from Walmart?“Our largest customer, Walmart Inc., represented approximately 21% of our net sales in 2025, 2024, and 2023.” | — | — | 2 | 1 | — | — | 2 | 1 |
APTVHow concentrated are Aptiv's sales among its largest customers?“Our ten largest customers accounted for approximately 56% of our total net sales” | — | — | — | — | 3 | 4 | 9 | 10 |
PNRHow much of Pentair's sales come from its single largest customer?“Our net sales to our largest customer represented approximately 18% of our consolidated net sales in 2025.” | — | — | — | — | 4 | — | — | — |
ADIWhat portion of Analog Devices' revenue is sold through distributors?“Sales to third-party distributors accounted for approximately 56% of our revenue” | — | — | — | — | — | — | 7 | — |
CORHow much of Cencora's revenue comes from Walgreens?“Walgreens and Boots together accounted for approximately 25% of our revenue in fiscal 2025” | 4 | — | 1 | 3 | 2 | 3 | 6 | 3 |
DLRHow much of Digital Realty's revenue comes from data centers it leases rather than owns?“These leased buildings accounted for approximately 14% of our total revenue” | 5 | 8 | 10 | — | 3 | 4 | 6 | 8 |
IDXXWhat share of IDEXX's revenue is recurring diagnostic revenue?“recurring diagnostic revenue, which is both highly durable and profitable, accounted for approximately 79% of our consolidated revenue” | 1 | 2 | 1 | 2 | 1 | 3 | 1 | 1 |
CIENHow much of Ciena's revenue comes from its largest cloud customer?“a cloud provider customer accounted for approximately 18% of our revenue” | 10 | — | — | 8 | 3 | 5 | 6 | 4 |
LHHow is Labcorp's revenue split between its two segments?“In 2025 and 2024, Dx and BLS contributed approximately 78% and 22%, respectively, of Revenues to the Company.” | — | — | 3 | 4 | 1 | 1 | 2 | 2 |
EXEWhere are Expand Energy's natural gas operations located?“Our operations are located in Louisiana and Texas in the Haynesville and Bossier Shales” | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
RSGHow many transfer stations does Republic Services operate?“We own or operate 255 transfer stations.” | 1 | 1 | 2 | 2 | 2 | 2 | 1 | 1 |
GWWWhat are Grainger's reportable segments?“Grainger's two reportable segments are High-Touch Solutions North America (High-Touch Solutions N.A.) and Endless Assortment.” | 1 | 1 | 5 | 1 | 2 | 1 | 1 | 1 |
IFFHow much of IFF's sales come from its 25 largest customers?“our 25 largest customers, a majority of which were multinational consumer products companies, collectively accounted for approximately 32% of our sales” | 1 | 4 | 1 | 7 | 1 | 1 | 1 | — |
How this was measured
- Each question was written for one company and paired with the exact sentence in its latest 10-K that answers it. The answer is located in the stored filing text, so “found” is checked mechanically, not judged by a model.
- The same filings were cut eight ways: fixed-size token windows and structure-aware chunks (which never cross a section and break on sentences) at 128, 256, 384 and 480 tokens, all within the embedding model’s 512-token window.
- Every chunk was embedded with gte-small (the model behind search). Retrieval combines meaning (exact vector similarity) and words (BM25), weighting words 1.5×, as the Ask page does. Every question searches all filings at once, so it competes with tens of thousands of passages from other companies.
- “Answer found” means the top five passages cover at least 80% of the answer sentence. “Text to read” is what a model would be given: the tokens in those five passages.
- Limits: 48 questions is a small sample (one question moves a score by about 2 points), they are mostly short factual look-ups, and the questions were drafted with an AI assistant from sentences in the filings. Long, multi-part answers would favour bigger chunks.
Parameter sweeps
Change one thing at a time. Quality is measured against labelled answers; cost is the chunks stored, tokens embedded, and context sent.
screening: Shortlisted configurations on a screening corpus. Used to choose what to run in full, never the final comparison. Hypothesis: There is no universal best size. Small chunks pinpoint facts but split answers; large chunks keep answers whole but cost more context and dilute the match. Structure-aware chunks should do better than blind token windows at every size.
Does size help or hurt?
Recall@5 as chunk size changes. One line per strategy and overlap.
- fixed
- structural
What does it cost?
Recall@5 against chunks. Ringed points can't be beaten on both.
- fixed
- structural
- on the Pareto frontier
All measured configurations
| Configuration | Recall@5 | MRR | Answer-span containment | Full-evidence rate | Chunks | Tokens embedded | Frontier |
|---|---|---|---|---|---|---|---|
| fixed.s480.o0 | 47.9% | 0.321 | 100.0% | 47.9% | 8,269 | 3.9M | Pareto |
| structural.max128 | 47.9% | 0.320 | 100.0% | 47.9% | 41,408 | 3.9M | |
| structural.max256 | 47.9% | 0.316 | 100.0% | 47.9% | 22,161 | 3.9M | |
| structural.max384 | 45.8% | 0.277 | 100.0% | 45.8% | 16,163 | 3.9M | |
| fixed.s384.o0 | 41.7% | 0.266 | 96.1% | 39.6% | 10,327 | 3.9M | |
| structural.max480 | 41.7% | 0.278 | 100.0% | 41.7% | 13,884 | 3.9M | |
| fixed.s128.o0 | 33.3% | 0.222 | 84.3% | 33.3% | 30,871 | 3.9M | |
| fixed.s256.o0 | 31.3% | 0.227 | 92.2% | 31.3% | 15,461 | 3.9M |