Our AI Agent Fabricated Statute Text. The Eval Layer Caught It.

Arguably the hardest failure mode to test for in production AI is not the hallucination that returns gibberish - it is the one that returns a plausible citation, a coherent legal explanation, and wrong law.

On July 11, 2026, our eval harness caught one of those. A smaller execution model was tasked with building a BC tenancy-law corpus module, with one explicit instruction: verify every fact against government sources before writing. It returned the correct operative values. It also invented a statutory discrepancy that does not exist in the statute, and wrapped it in a convincing legal explanation drawn from a section of the RTA that says nothing of the sort.

Here is what happened, what caught it, and why the architecture that stopped it is what we mean by "production-grade" for regulated AI.

The task: a sourced corpus for BC tenancy service rules

The corpus module tracks how documents are deemed received under BC residential tenancy law - specifically the day offsets under RTA s.90 that govern when a notice is legally considered to have arrived after you serve it.

These values carry real consequences. A rent-increase notice served by mail on the wrong day is void even if every other part of the notice is correct. A two-day error against the mail offset can erase an otherwise valid landlord-use eviction. The stakes justified the verification requirement.

The model's assignment was specific: read the BC government's serve-notice guidance page and RTB Policy Guideline 12, extract the deemed-receipt values, and write a corpus entry for each service method with source URL, retrieval date, and effective period.

What the model produced

The operative values were correct.

In-person delivery: 0 deemed days (received the same day). Door posting or mailbox: 3 days. Mail, registered or regular: 5 days. Email with prior written consent: 3 days. These match the gov.bc.ca serve-notice page exactly.

What the model also produced was a fabricated discrepancy. It claimed that RTA s.90 itself - the statute text, not the policy guideline - said door-posting adds 0 days and mail adds 7 days, and that the values it had returned therefore differed from the statute. It then supplied a plausible legal explanation: regulations under s.89 had modified the statutory defaults.

The statute says no such thing. The phantom discrepancy, the phantom 7-day mail rule, and the phantom s.89 regulatory override appear nowhere in the RTA.

Why this is the failure mode that matters

A hallucination that returns obvious garbage is caught early - often by the person who ran the query. The model returns "mail adds 23 days" and someone checks.

The failure mode that escapes early detection is one where the wrong claim is wrapped in a structure that matches how correct claims look. The model had already retrieved the correct values. It knew the statute reference (s.90). It knew the policy guideline reference (GL12). It understood that a plausible discrepancy between a statute and a derived guideline would require an explanation. So it invented one - complete with a secondary statute reference (s.89) and a mechanism (regulations modifying the statutory defaults).

To a reader who had not read the statute, that explanation was indistinguishable from accurate legal analysis.

I do not think better prompting would have prevented this. The model fabricated a citation that looked exactly right because it has seen a great many correct legal citations. The failure is not ignorance of what a legal citation looks like; the failure is a tendency to produce plausible outputs when the truth is uncertain, rather than flagging the uncertainty. That tendency cannot be prompted away. What it requires is an independent verification layer.

What caught it

The grading layer.

This is how we build, not a feature we sell: our development harness runs two models in sequence. A smaller execution model produces the output. A stronger reviewing model - working from a different instruction set and given the same source material - is told to verify every factual claim independently before accepting the output as graded-pass.

The reviewing model flagged the s.90 citations as unverifiable. It could not find the door-posting = 0 / mail = 7 text in s.90. It could not find the s.89 regulatory mechanism. It returned the finding as a grading failure, naming the specific claims it could not corroborate.

That is the adversarial layer doing its job: not checking that the output is internally coherent, but checking that the output's claims can be found where the output says they can be found.

Human-in-the-loop confirmed the fabrication

The grading flag triggered a human check. I read RTB Policy Guideline 12 §N (the June 2025 edition, PDF from gov.bc.ca) against the model's output.

GL12 §N states directly:

"a record is considered or 'deemed' received: if given or served by mail (ordinary or Registered Mail/Express Post with signature option), on the fifth day after mailing it; if given or served by fax, on the third day after faxing it; if given or served by email, on the third day after emailing it; if given or served by attaching a copy of the record to a door or other conspicuous place, on the third day after attaching it; and if given or served by leaving a copy of the record in a mailbox or mail slot, on the third day after leaving it."

Mail: 5th day. Door posting: 3rd day. Email: 3rd day. No discrepancy with s.90. No regulations under s.89.

The operative values the model returned were correct all along. The phantom discrepancy was not an error in the values - it was a fabrication layered on top of correct values, making them appear to require an explanation they did not need.

The fix: provenance on every value, adversarial evals in CI

The corpus file (apps/perch/corpus/service-rules.ts) now carries three things that were not present in the draft version.

The verbatim GL12 §N quote above appears as a block comment in the file header. Any future model reading this corpus sees the source text alongside the values, rather than being asked to reconstruct it from a URL.

Every corpus entry carries provenance metadata: sourceUrl, retrievedAt, and effective date range. There is no fallback to a plausible value. Any query against an entry flagged unverified throws a typed error before it can reach any output a user sees.

The eval suite now runs 157 automated tests in CI on every change headed to production. The adversarial suite - 36 cases - specifically targets input guards: out-of-range years throw; negative rent values throw after this round found the helper silently accepting them instead of rejecting them. The goal of the adversarial suite is to find the inputs where the helpers return a plausible answer when they should return an error.

What demos cannot catch

The failure I described above would have passed any functional test you could write before the fact. The values were correct. The module would have returned the right answer for any query about mail service or door posting.

Demos test the happy path. Adversarial grading tests whether the model is telling the truth about why it produced the happy path.

In a regulated domain - tenancy law, financial services, healthcare compliance - the citation and the explanation often carry as much weight as the number. A tenant's ability to challenge an eviction notice depends on whether the deemed-receipt rule was correctly applied and correctly explained. An advisor building on top of an AI corpus trusts that corpus's citations, not just its values. If those citations are fabricated, the trust is misplaced, and there is no visible signal of the misplacement until the moment it causes harm.

The unfashionable answer, when you are building AI for regulated use, is to write a lot of tests and to grade adversarially. "Plausible and wrong" is not a prompt engineering problem. It is an architecture problem.

What production-grade actually means

Production-grade for regulated AI means the system can prove its claims, not just produce them.

That requires three things in combination:

A deterministic corpus layer with source URLs and retrieval dates per value - the layer that carries the ground truth that the generative layer is expected to reflect.

A grading architecture where a stronger reviewing model independently verifies that the generative model's citations are accurate against the sources provided - not internally consistent, not plausible, but verifiable.

An adversarial eval suite that runs in CI before anything ships and specifically hunts for the inputs where the system produces a confident answer when it should refuse or throw.

The interesting part is not the corpus. The interesting part is the grading layer that checks whether the model's claims about the corpus are themselves true. That is the layer the phantom s.89 regulatory mechanism did not survive.

If you are deploying AI in a regulated domain

The question is not whether your model can produce the right answer in a demo. Smaller execution models produce correct answers frequently - the fabrication we caught yesterday was surrounded by correct answers on every side. The question is whether your architecture can detect the failure mode where the model produces a correct answer and a fabricated explanation for why the answer is correct.

That detection requires a grading layer with independent source access, a corpus that carries its own receipts, and adversarial evals designed specifically to surface the lies that look like analysis.

If you are evaluating that architecture - or deciding whether you need to build one - we work on this problem. Book a discovery call.