How to chunk web pages for retrieval

Most retrieval problems get diagnosed as embedding problems or reranking problems. A surprising share of them are chunking problems, and chunking is the part almost nobody revisits after the first implementation.

TLDR

• Boundaries beat size. A chunk that starts and ends where an idea does retrieves better than any tuned token count.
• Web pages come with boundaries already: headings. Use them.
• Never split a table or a code block. Half of either is worse than none.
• Prefix chunks with their heading path, so a chunk says what it is about before it says anything else.
• Remove boilerplate first, or every chunk carries a slice of the navigation and pages stop being distinguishable.

Why fixed-size chunking fails on web pages

Split every 512 tokens and you get chunks that are the right size and the wrong shape. The failure is specific:

A section explaining a concept gets cut in half. The first chunk sets up a problem and stops before the answer; the second states the answer with no indication what it is answering. Both embed cleanly. Both retrieve for related questions. Neither is useful, and nothing in your pipeline reports a problem.

Fixed-size chunking works acceptably on flowing prose, which is why it became the default in tutorials that use novels and essays as examples. Web pages are not flowing prose. They are structured documents, and the structure is the thing you are discarding.

Use the boundaries the page already has

An author writing a page with ## headings has already told you where each self-contained idea begins and ends. That is a better boundary than any heuristic, and it is free.

The basic algorithm:

1. Split at headings, keeping the heading with its section
2. If a section exceeds your ceiling, split it further at paragraph boundaries
3. If a paragraph alone exceeds the ceiling, then and only then fall back to sentences
4. Merge very short sections into the next one, so a two-line heading is not its own chunk

Notice size only enters at step two, as a constraint rather than a target. That is the right relationship.

Give each chunk its address

The single cheapest improvement available: prefix every chunk with the page title and its heading path.

Billing > Overage > How overage is calculated

Overage applies once a workspace has spent its included
credits for the period...

Retrieved alone, the raw section might never mention billing at all, because the heading already said it and prose does not repeat what the structure establishes. The prefix restores that context for a handful of tokens, and it helps twice: the embedding has more to match on, and the model reading it knows where it came from.

Keep tables and code intact

Two things must never be split, regardless of size limits.

Tables. Half a table is rows without headers, which is data without meaning. If a table is too big for a chunk, put it in its own chunk and let it be oversized, or repeat the header row across parts.

Code blocks. A fragment of a code sample is actively harmful, because it looks complete enough to copy. Keep the block whole and, if it is enormous, chunk around it rather than through it.

Both are cases where breaking your own size rule is correct.

Strip the boilerplate before you chunk, not after

If navigation, cookie banners and footers are still in the page when you chunk, every chunk gets a slice of them.

The obvious cost is wasted tokens. The real cost is subtler: because site chrome is identical across every page of a site, chunks from unrelated pages become measurably similar to each other. Your retriever starts returning the wrong page for the right question, and the reason is not in your retriever.

Extraction is a prerequisite for chunking, not an optimisation to add later.

Overlap is a patch, not a strategy

Overlapping chunks by some percentage is standard advice, and it exists to soften the damage of arbitrary boundaries. With structural boundaries you need much less of it.

A sentence or two of overlap is worth keeping, since ideas occasionally straddle a heading. Large overlaps mostly inflate your index and return near-duplicates that crowd out genuinely different results.

Retrieve small, supply large

A useful pattern once the basics are right: embed and retrieve at chunk level, then give the model the surrounding section rather than the chunk alone.

Small chunks retrieve precisely. Larger context answers better. Doing both means storing the page structure alongside the chunks so you can expand a hit to its parent section on the way into the prompt.

This is usually a bigger quality win than switching embedding models, and it costs nothing at query time.

A checklist

• Extract to clean markdown first
• Split on headings, then paragraphs, then sentences, in that order
• Prefix with the heading path
• Never split tables or code
• Small overlap only
• Keep the page so you can expand a hit to its section

If your answers are wrong and you have been tuning the retriever, print twenty of your chunks and read them. The problem is usually visible in about a minute, and it is usually here.

Scrape and crawl return clean markdown with the boilerplate already gone and the headings, tables and code blocks intact, which is what makes structural chunking possible in the first place. One credit a page, failures never billed, 500 free credits on a work email.