Artificial Intelligence in Finance

Getting Reliable Structured Outputs from LLMs: Validation, Retries and Constrained Decoding

The Problem with Syntactic Perfection

The current state of LLM-based data extraction is defined by a reliance on schema enforcement. Modern APIs from providers like OpenAI and platforms like vLLM, which leverage technologies like xgrammar, allow developers to supply a Pydantic model or JSON schema to guide generation. In theory, this guarantees that the model will return a response that adheres to the defined structure.

However, a recurring failure point in these systems is the "semantic gap." A model might generate a perfectly valid JSON object that passes all schema validation—every key is present, the data types are correct, and the values are within allowed sets—yet the information itself may be fundamentally incorrect. For example, in an invoice processing scenario, a model might correctly identify the "invoice_id" and "currency," but calculate a "total_cents" value that contradicts the printed text on the invoice. If an invoice indicates a total due of EUR 1,250.00, a model might incorrectly output 12,500 cents (125 EUR) instead of the correct 125,000 cents. Because the value 12,500 is a valid integer, no JSON parser or schema validator will flag it as an error. This highlights why validation must be treated as a two-stage process: structural validation followed by source-of-truth verification.

The Three-Layered Architecture for Reliability

To mitigate these risks, developers are increasingly adopting a three-tiered defense strategy: strict validation, intelligent retries, and constrained decoding.

The first layer, validation, requires the application to own the contract. Using libraries such as Pydantic, developers can define a rigid schema with strict mode enabled. By setting strict=True and extra="forbid", the application ensures that the model cannot perform "coercion"—such as turning a string into an integer—or introduce unauthorized fields. This creates a secure "gate" through which all model outputs must pass.

The second layer involves an acceptance function, accept(), which acts as an arbiter between the parsed model object and the source document. This function performs a character-by-character or logical check to ensure that the extracted figures are derivable from the input text. If the "total_quote" field does not match the source text, or if the "total_cents" does not correlate with the figures present in the document, the entire object is rejected. This prevents the downstream propagation of hallucinations or calculation errors.

The third layer, constrained decoding, moves the intervention upstream. Rather than asking the model to generate text and then checking it, constrained decoding masks the model’s token output space in real-time. By restricting the generator to only select tokens that maintain structural validity, the model is physically prevented from generating invalid JSON, code fences, or stray text.

Getting Reliable Structured Outputs from LLMs: Validation, Retries and Constrained Decoding

Chronology and Operational Logic

The lifecycle of a robust extraction pipeline follows a specific, repeatable sequence. When a document enters the system, the pipeline first performs optical character recognition (OCR) or text extraction. It is essential to inspect this raw data for quality before the LLM is even invoked; a digit misread by the OCR engine will propagate through the entire pipeline regardless of the model’s performance.

Once the text is ready, the pipeline executes the generation request. If a shape error occurs (i.e., the output is not valid JSON), the system initiates a retry. Crucially, the retry logic must be bounded. A common standard is a maximum of three attempts. In this framework, the first attempt is the primary request, followed by two potential regenerations. The system provides the model with specific, localized feedback from the validator—such as "total_cents does not match the quoted line"—to guide the model toward a correction.

However, if the error is semantic—meaning the JSON is valid but the data is wrong—the system must trigger a terminal "rejected" state. Retrying a semantic error is often counterproductive, as the model has already demonstrated it cannot derive the correct answer from the provided context. By distinguishing between structural failures and logical errors, developers can conserve latency, reduce API costs, and prevent the "infinite loop" scenarios that plague poorly designed agents.

Broader Implications and Industry Best Practices

The transition toward automated data extraction has significant implications for sectors such as finance, logistics, and legal services. As organizations move away from manual data entry toward LLM-driven automation, the cost of an error—such as an incorrectly processed invoice—can be substantial.

Industry analysis suggests that the "brute force" approach of simply increasing model size is not a viable path to reliability. Instead, the focus has shifted toward "small-model engineering," where developers use specialized, smaller models constrained by rigid programmatic frameworks. By keeping the contract plain and avoiding overly complex Pydantic features that may not survive the conversion into a provider’s schema subset, developers create a more resilient and portable system.

Furthermore, the "needs_review" category is a vital component of a mature architecture. When an LLM encounters a missing piece of information, it should be prompted to return a null value rather than hallucinating or guessing. A system that explicitly routes these null responses to a human-in-the-loop (HITL) process is significantly more reliable than one that forces the model to invent data to satisfy an schema requirement.

Conclusion: The Future of Deterministic AI

The integration of LLMs into production environments requires a move away from the "black box" mentality. By treating the model as a fallible source of content that must be constrained, validated, and audited, developers can achieve a high degree of confidence in automated outcomes. The goal is not to eliminate human oversight, but to reduce the volume of mundane tasks while ensuring that the data reaching the enterprise systems is accurate. As providers continue to improve the support for structured outputs and constrained decoding, the focus will increasingly shift from "how do we get the model to speak" to "how do we ensure the model speaks the truth." This shift toward deterministic AI pipelines is the next frontier for software engineering in the age of generative models.

Related Articles

Back to top button