Use Case: When AI Reads Your Forms and Quietly Gets the Numbers Wrong

Here’s a request that lands on a lot of engineers’ desks, especially anyone working near insurance, banking, or lending. A client has a mountain of forms, insurance claims, account applications, statements, and they want an AI that pulls the right values out of them: this policy number, that claim amount, this date, correctly, every time. It sounds like the document-chatbot problem from my last post. It isn’t quite. A support bot answers questions from prose. This has to read a form and get exact numbers right, and in insurance or banking a wrong number is a mispaid claim or a compliance failure, real money on the line.

So you reach for the obvious tool: RAG. It’s brilliant on paragraphs, chop the text into chunks, find the matching chunk, hand it to the model, get a grounded answer. And then you point it at a form and it starts quietly getting numbers wrong. It answers “what’s the policy number?” with the account number. It swaps one amount for another. Nothing crashes; it just returns confident, incorrect values, which is the worst kind of failure here.

Let me walk through how I’d actually build this. The surprise is that the fix is almost never a bigger or smarter model. The whole battle is won or lost in one unglamorous step, chunking, how you cut the document into pieces before you ever store it. Naive chunking that works fine on prose destroys a form. So I’ll show why it breaks, the approaches that fix it, and, honestly, the side effect and accuracy cost of each. (I’ll assume the text is already pulled off the page, getting text off a scan is a separate job. Here we care about what happens after: how you cut it up.)

The one thing that makes a form different

Quick reminder of why forms aren’t prose. In a paragraph, meaning flows in a line, you can cut it almost anywhere and each piece still makes sense. In a form, meaning is a pairing: a label and its value belong together, and a table cell only means something next to its row and column headers.

Policy Number:

POL-88231

Claim Amount:

$4,500

Deductible:

$500

"$4,500" only means "claim amount" because it sits next to that label. "$500" is the deductible for the same reason. Separate a value from its label, or a number from its header, and the meaning is gone. Hold this: on a form, a value and the thing that names it must never be split apart. That single rule explains every chunking decision below.

Why the usual chunking breaks it

The default chunking strategy is dead simple: cut the text every ~500 words, maybe with a little overlap. On prose, fine. On a form, watch what it does.

Naive chunking (cut by size)

...Deductible: [cut here]

$500 Claim Amount: $4,500 Policy...

"$500" got separated from "Deductible". Now which amount is which?

Structure-aware chunking

Deductible = $500

Claim Amount = $4,500

Each label stays glued to its value. Meaning preserved.

Left: cutting by size slices right through a label-value pair, orphaning "$500" from "Deductible". Later, when someone asks "what's the deductible?", retrieval finds a chunk with a lonely "$500" or a jumble of amounts, and the model guesses. Right: cut along the form's structure instead, keep each pair whole, and the value never loses its label. Same document, wildly different reliability.

That’s the whole problem in a nutshell: naive, size-based chunking treats a form like prose, and forms aren’t prose. So how do you chunk them properly? There are a few approaches, and each buys accuracy at a different price.

The chunking approaches, with their real trade-offs

Here’s the menu, from naive to what I’d actually reach for, each with its honest side effect and its effect on accuracy.

Fixed-size chunking

low accuracy on forms

Cut every N words, ignore structure. The default that works on prose.

Side effect: splits labels from values and rows from headers, so retrieval returns orphaned numbers and the model mismatches fields. Fine for paragraphs, actively harmful for forms.

Field-aware chunking (one pair per chunk)

high on simple forms

Chunk along the form's structure: each label-value pair (or small group of related fields) becomes its own chunk, kept whole.

Side effect: you need to detect the structure first (which requires knowing the form's layout), and very tiny chunks can lose surrounding context. But for key-value forms, accuracy jumps because nothing gets orphaned.

Table-aware chunking (one row per chunk)

high for row lookups

For tables, index each row as a chunk, carrying its column headers with it, so a row's cells stay tied to what they mean.

Side effect: great for "what's in row X?" questions, but it can lose cross-row context, "which row has the highest amount?" needs the whole table, not one row. Research on structure-aware table chunking reports strong faithfulness gains, but you trade away some table-wide reasoning.

Parent-child chunking

best all-rounder

Keep two sizes: small child chunks (a single field or row) for precise finding, linked to a larger parent chunk (the whole section or table) for understanding. You search on the small ones, but hand the model the big one.

Side effect: you maintain two index layers and it's more work to update when a doc changes. But it resolves the core tension, precise retrieval and enough context, which is why it's the single biggest quality win for most setups.

Turn it into structured data (skip RAG)

highest, when it fits

If the form always has the same fields, don't chunk-and-search at all, extract the fields into a proper table (policy_number, amount, ...) and query them directly.

Side effect: only works when fields are known and consistent, and you lose RAG's flexibility for free-form questions. But when it fits, it's the most accurate and verifiable option, exact values, no fuzzy retrieval, no hallucinated numbers.

Five approaches, one theme: the more you respect the form's structure, the higher your accuracy, at the cost of more setup. Fixed-size (1) is easy and wrong for forms. Field- and table-aware (2, 3) fix the orphaning but need structure detection. Parent-child (4) is the workhorse. And sometimes (5) the right move is to stop treating it as a search problem at all.

The golden rule behind all of them

Every good approach above is really following one principle. If you remember nothing else, remember this:

Never split a value from the thing that names it. A number without its label is noise. Chunk along the form's structure, label-with-value, row-with-headers, so every piece that gets stored can still be understood on its own.

This is the whole art of chunking a form. Prose chunking asks "where's a natural sentence break?" Form chunking asks "what's the smallest piece that still carries its own meaning?" For a form, that piece is a labelled field or a headed row, never an arbitrary slice of text.

A closer look at the workhorse: parent-child

Because parent-child chunking is the one I’d reach for most, here’s how it actually works, and it’s a neat idea. You store two linked versions of the content: tiny pieces to search with, and big pieces to answer from.

You search on the small child chunks (each a single precise field), so retrieval is sharp, "claim amount" matches exactly the right little piece. But you answer from the parent (the whole section), so the model has the surrounding context and doesn't misread. Small chunks for finding, big chunks for understanding. That combination is why it beats using one chunk size for both.

The trick that quietly does a lot: metadata

One more technique that pairs with all of these, and it’s easy to overlook. When you store each chunk, tag it with metadata, extra labels about what it is: which form, which field, which section. Then retrieval can filter before it even searches.

form: ACORD-25

tag every chunk with the form type

field: claim_amount

and which field it holds

filter first

Query "claim amount on the auto claim" filters to just those chunks, then searches. Faster and far more precise.

Metadata turns a fuzzy search into a targeted one. Instead of searching all chunks and hoping the right field floats up, you first narrow to "claim-amount fields on this form type," then retrieve. It's cheap to add and it sharply cuts the wrong-field mistakes that plague form RAG. On structured documents, good metadata is often worth more than a fancier model.

Which one should you actually pick?

The honest decision guide, matched to your forms and your risk.

Your situation	Reach for	Because
Simple, consistent key-value forms	Field-aware chunking	Keeps each pair whole; big accuracy win, low effort
Lots of tables, row-level questions	Table-aware (row) chunking	Ties each row's cells to their headers
Mixed forms, varied questions	Parent-child chunking	Precise finding plus enough context; the best all-rounder
Same fields every time, exact values matter	Extract to structured data, skip RAG	Most accurate and verifiable when the shape is fixed
Any of the above, high stakes	+ metadata + a confidence check	Filter to the right fields, and flag uncertain ones for a human

No single winner, it depends on your forms. But the through-line is constant: the accuracy problem with form RAG is almost always a chunking problem, not a model problem. Fix how you cut the document, and the same model that was getting numbers wrong starts getting them right.

Want the real research?

If you want to go deeper than this overview, these are solid primary sources on chunking structured and tabular data for RAG:

Structure-aware chunking for tabular data, on why row-and-header-aware chunking beats naive splitting, with faithfulness numbers: Structure-Aware Chunking for Tabular Data in Retrieval-Augmented Generation.
Semi-structured table question answering, a look at answering questions over messy real tables: ST-Raptor (arXiv 2508.18190).
Adaptive retrieval over tabular formats, on structured querying instead of blind chunk search: SQuARE (arXiv 2512.04292).

The takeaway

When RAG gets a number wrong on a form, the instinct is to blame the model or reach for a bigger one. Nine times out of ten, the real fault is upstream, in how you chopped the document. Naive size-based chunking treats a form like a paragraph and slices right through the label-value pairs that carry its meaning, leaving retrieval with orphaned numbers to guess from.

The fix isn’t fancier AI, it’s chunking that respects the form’s shape: keep each label glued to its value, each row glued to its headers, use parent-child so you can find precisely and answer with context, tag everything with metadata so you can filter to the right field, and when the form is fixed and the stakes are high, skip fuzzy search and pull the fields into a real table. Every one of these has a cost, more structure detection, more index layers, less flexibility, but they all buy the same thing: numbers that come out right. On a form, accuracy was never really about the model. It was about not cutting the meaning in half before the model ever sees it.