How to Chunk PDFs for RAG Without Losing Tables and Context?

Article

Why PDFs Break RAG Retrieval

You ask your assistant for the refund threshold from last quarter’s policy PDF and it confidently gives you the wrong number. You check the source and the answer was right there in a table on page nine. The problem was not the language model. The retriever never brought back that table intact, so the model guessed from fragments.

This happens because PDFs store where words appear on a page, not what they mean. Columns, headers, footers, footnotes, and scanned images can all scramble the reading order when you extract text. If you then cut that extracted text into fixed-size pieces, you often slice a paragraph in the middle and scatter table rows across unrelated chunks.

Good PDF chunking for RAG is therefore less about picking a perfect size and more about preserving usable units. A usable chunk lets a reader, or a model, understand it without seeing the whole document. That means keeping the section heading, keeping tables together, and keeping enough context to know what the numbers refer to.

What to Preserve Before You Split Anything

Before you decide where to split, decide what each chunk needs to carry to be interpretable on its own. For most business PDFs like reports, manuals, contracts, and papers, that means the document title, the section heading, and the immediate context around a table or figure. Without those anchors, a chunk that says net revenue increased by 12 percent is useless because you do not know which product, year, or region it describes.

It also helps to preserve location and structure metadata outside the text itself. Page numbers, section paths, table titles, and whether the chunk came from body text, a caption, or a table help you debug retrieval later and show better citations to users. You do not need a complex schema to start. Consistency matters more than completeness.

Keep the document title and section path with every chunk

Keep each table title and surrounding paragraph together with the table

Record page number and source file name for citations and debugging

Strip repeated headers, footers, and page numbers from the chunk text itself

Flag scanned pages that need OCR so you do not index empty or garbled text

A Practical Process to Chunk PDFs for RAG

You do not need to rebuild your whole pipeline to get better PDF chunks. Pick one representative PDF that actually matters to your users and walk through it end to end. That one document will teach you more than tuning settings across hundreds of files.

Use this process as a starting point and adjust it for your documents. The goal is to produce fewer, cleaner chunks where each one answers a plausible user question on its own.

If a chunk still confuses a colleague who has not read the full PDF, it will likely confuse retrieval too. Rewrite the boundaries or enrich the context until it stands alone.

Extract with structure: use a PDF parser that preserves headings, reading order, and tables, and export to markdown or HTML where possible

Clean the layout noise: remove running headers, footers, and page numbers, and fix two-column text into correct reading order

Run OCR only where needed: detect scanned pages and process them separately so you do not degrade clean digital text

Split by logical sections first: use headings as natural boundaries, then split long sections into smaller passages only if needed

Treat each table as one unit: keep the full table plus its title and the paragraph that explains it in the same chunk

Attach minimal metadata: source file, page range, section heading, and content type such as text or table

How to Handle Tables Without Creating Junk Chunks

Tables cause most PDF failures in RAG because a single row without column headers is meaningless. If you split a pricing table across three chunks, the retriever might return quantity and discount without the product name. The model then either refuses or invents the missing link.

The safest default is to keep small to medium tables whole in one chunk. Convert them to a simple format that preserves rows and columns, such as markdown tables or a clean row-by-row list, and include the table title and the sentence before and after the table. That surrounding text usually explains units, time periods, and exceptions that are not inside the cells.

For very large tables that do not fit sensibly in one chunk, do not slice them blindly by row count. Group rows that answer the same type of question, repeat the header and table title in each derived chunk, and add a one-line description like returns by region in USD for Q3. If a table is purely reference data that users filter rather than ask about, consider summarizing it in text and keeping the full table for citation while letting search focus on the summary.

How to Tell If Your PDF Chunks Are Working

You cannot judge chunk quality by looking at settings alone. Judge it by whether real questions retrieve the right pieces. Create a small test set of ten to fifteen questions that your PDF should answer, including at least three that require table values and two that require context from a specific section.

Run those questions through retrieval only, before generation, and inspect what comes back. A healthy result is one where the correct table or paragraph appears in the top results and reads clearly without opening the PDF. If you see fragments, duplicated headers, or rows without context, fix the chunking rather than adding more documents.

Does the top chunk contain the full answer plus its heading or table title

Could someone understand the chunk without opening the source PDF

Are tables returned whole with headers, units, and time period intact

Are citations pointing to the correct page and section

Start Small, Then Scale What Works

PDF chunking improves fastest when you treat it as document preparation, not just a size parameter. Preserve headings, keep tables together, attach light metadata, and test with real questions. Those four habits solve most of the vague answers, missing numbers, and wrong citations that teams blame on the model.

As a next step, take one high-value PDF and reprocess it with the workflow above, then compare retrieval before and after on your test questions. Whether you work manually or explore a tool like ChunkCraft, that before-and-after comparison will show you what to standardize across the rest of your collection.