Detecting LLM Hallucinations Without a Slow Second Model Call: A Reference Architecture

The common objection to real-time hallucination detection is latency. If verifying an LLM output takes 3 seconds, you have not improved your AI workflow - you have just added a slow step before the broken output reaches your users.

The objection is valid. The 3-second assumption is not.

The design below is built to keep verification small next to the model call itself. We have not published latency measurements for it. Whatever you build, measure it on your own documents and workflows. The workflows where this matters are research summarization, document Q&A, compliance screening and report generation, where users already expect a response in a second or more.

This post explains the architecture that keeps verification fast.

The problem with naive verification

The obvious implementation of hallucination detection is a second LLM call. Produce the output, then ask a second model "is this faithful to the source?" Two sequential LLM calls can add seconds of verification overhead. This is the approach most teams attempt first, and it explains why they conclude real-time verification is impractical.

Two problems with this approach:

First, the verifying model can hallucinate in exactly the same way as the original. You are not measuring faithfulness - you are measuring whether one stochastic process agrees with another.

Second, latency is a product of the architecture, not the task. Two sequential LLM calls are slow because they are sequential. The solution is to not use LLM calls for verification.

The two-stage verification architecture

Stage 1: Claim extraction

Extract atomic factual claims from the LLM output. A claim is a unit of information that is independently verifiable: "The company reported $4.2B in revenue in Q3 2024" is a claim. "The company performed well last quarter" is not - it is an interpretation.

A small fine-tuned language model is enough for claim extraction. Small models are fast, so extraction should not dominate the latency budget.

Stage 2: Source cross-reference

For each extracted claim, retrieve the most relevant passages from the source document set using vector similarity search, then score the claim against those passages using a specialized entailment model.

The entailment model is not a general-purpose LLM. It is a fine-tuned NLI (natural language inference) model optimized for the entailment task: given a claim and a passage, does the passage support, contradict, or fail to address the claim? These models are small and fast per claim, and claims can be scored in parallel.

The vector retrieval step and the entailment scoring step run concurrently where possible.

Output: verification record

Aggregate the per-claim scores into a document-level verification score, flag any unsupported or contradicted claims, and write the result to the audit log with a cryptographic hash. This step is cheap because it is computation, not inference.

The multi-second verification you were afraid of is an artifact of the wrong architecture. Sequential LLM calls are slow. This pipeline avoids them.

What the verification record contains

For each verified output, the record includes:

• Verification score (0.0 to 1.0): The proportion of extracted claims that are supported by source documents. A threshold you choose can trigger a compliance flag.
• Claim list: Each atomic claim with its individual entailment score and the source passage(s) used.
• Flagged claims: Claims scored as unsupported or contradicted, with the relevant source context.
• Metadata: Model version, timestamp, document set hash, verification model version.
• Chain hash: SHA-256 hash linking this record to the previous one in the audit log.

This record is what makes the audit trail useful. Not just "this output was verified" but "here is exactly which claims were verified, against which source passages, with which scores."

Accuracy and latency: what to measure

We do not publish a detection rate or a latency figure for this design, because both depend on your documents and your task. Before you rely on any verification layer, measure these on your own workflow:

• Claim extraction recall: how many of the verifiable claims in a typical output are found. Implicit claims and comparative statements are hard to split into atomic form, so expect misses there.
• Entailment accuracy: how often the scoring model agrees with human labels on claim and passage pairs from your own documents.
• Hallucination detection rate: the share of known-bad outputs the whole pipeline flags, compared with your current baseline.
• Added latency: the median and the slow tail of the verification step, next to the model call it follows.

No verification system catches everything. The goal is a lower and measured risk, with a complete audit record, instead of an unmeasured one.

This is a reference design, not a product claim. In a pilot we measure your baseline on your own workflow first. If you want to talk through how it applies to your LLM workflows, the conversation takes 20 minutes.