Document Data Validation: Guide & Interactive Validation Lab

Worked example: a confident extraction with a wrong total

The lab starts with a fictional invoice showing a CAD 1,000.00 subtotal, CAD 50.00 tax amount and CAD 1,100.00 total. The supplied extraction confidence is 99%, but the arithmetic differs by CAD 50.00. With the example CAD 0.02 tolerance, the arithmetic rule fails and the proposed record goes to review. The confidence check cannot cancel that failure.

Switch off total reconciliation and the same invoice can reach the next approval gate under the remaining rules. Run the fixed sample set to see the missed error. This is a deliberate teaching exercise, not a recommendation to disable a control. The CAD amounts, identifiers, confidence values and rounding contract are synthetic; tax-rate validation, payment authority and real supplier verification are outside the lab.

A correctly extracted value is not automatically valid

Extraction asks whether the system captured what appears on the document. Validation asks whether that captured value is complete, plausible, consistent and appropriate for the intended action. Those are different questions. An invoice can clearly show an incorrect purchase-order number; perfect extraction would faithfully reproduce the wrong value.

Place validation between extraction and the acceptance gate in the wider document processing workflow. Keep proposed values in a staging state until the critical rules and required approvals have completed.

Classify fields according to business consequence

Not every value deserves the same validation effort. Identify the fields that determine identity, amount, terms, routing or authority. A mistaken customer identifier or payment total can affect a material action; an optional note may be safe to leave blank or present only for reference.

For each field, document whether it is required, the permitted format and range, which source is authoritative, how uncertainty is detected and what happens when the rule cannot run. This becomes the validation contract for implementation and testing.

Layer simple rules before complex judgment

Run inexpensive deterministic checks before asking a model or person to interpret an ambiguous case. A missing required field, invalid date, impossible total or known duplicate can be identified without a subjective decision. Early checks also prevent weak inputs from contaminating later steps.

Treat unavailable evidence differently from a failed rule. If the customer database is offline, the workflow does not know that the proposed customer is invalid; it knows the comparison could not be completed. That distinction determines whether to retry, review or reject.

Use confidence as a routing signal, not a correctness certificate

Field confidence can help identify low-quality scans and uncertain extraction, but the same threshold should not be applied blindly to every field or document type. A high-confidence total still needs arithmetic validation; a moderately confident note may be acceptable when it cannot trigger an action.

Choose thresholds using representative documents and the consequence of error. When evidence remains uncertain or only a person may accept the action, use a purposeful human-in-the-loop automation decision.

Turn each failed rule into an efficient review task

A reviewer should not have to rediscover why an item stopped. Display the original document region, proposed normalized value, failed rule, relevant source-system value and permitted actions together. Preserve the proposal and final correction so the outcome can be explained and recurring patterns can be measured.

Avoid silently overwriting extracted data. An approved deterministic normalization can create a separate normalized value; a material correction should retain who or what made the change, the reason and the source that supports it.

Test validation with normal, boundary and exceptional documents

A useful test set includes ordinary documents, known bad inputs and values near every material boundary. Test missing fields, conflicting totals, unknown entities, duplicates, damaged scans and temporarily unavailable source systems. Record the expected rule outcome before running the test.

After launch, monitor critical-field correction rate, failed-rule frequency, reviewer disagreement, auto-accept rate and errors discovered downstream. Review thresholds when document layouts, suppliers, policies, integrations or consequences change—not simply because the model reports a different confidence distribution.

Build validation around the decision the document supports

Start with one document type and write the field-level acceptance matrix before tuning extraction. The matrix makes implementation testable and prevents a visually impressive extraction demo from becoming an unreliable operating process.

Use the lab’s exports as a starting structure, not production configuration. Replace synthetic cases with permissioned, representative examples labelled by your team; add field-level rules, known supplier variations, real duplicate keys, approval ownership and unavailable-system tests. Keep the evidence labels independent of the rule settings so a looser policy cannot make its own mistakes disappear.

OpSmith’s document processing automation work connects extraction to business rules, source evidence, review and the verified destination update.

Frequently asked questions

Sources and further guidance

These official references provide relevant technical, privacy, risk-management or accountability guidance. OpSmith applies the useful principles to the operation of one business workflow.

• Evaluate document extraction against labelled test dataGoogle Cloud
• Document AI validation and correctionGoogle Cloud
• Enterprise Document OCR and image-quality analysisGoogle Cloud
• PIPEDA fair information principle: limiting collectionOffice of the Privacy Commissioner of Canada