Core Concepts

Eval-first

Eval-first means verification is not a post-deployment afterthought. Every output from your LLM goes through a structured evaluation step before it reaches a user. The platform measures accuracy against source documents, flags uncertain or unsupported claims, and logs the result - all in the same request cycle.

This is different from monitoring, which tells you something went wrong after the fact. Eval-first catches the problem before the output is delivered.

The SAFE framework

Fact AI Lab is built around the SAFE framework: Scope, Audit, Fit, Eval.

Scope defines the boundaries of what the LLM is allowed to do in a given workflow - which documents it can reference, which claim types are permitted, and what constitutes an out-of-scope response.

Audit is the tamper-evident log of every verified interaction. Each entry records the prompt, the retrieved context chunks with source citations, the model version and temperature, the raw LLM output, the verification score, any flagged claims, and a cryptographic hash linking to the previous entry. Logs can be exported as PDF or JSON for regulatory examination.

Fit measures how well the LLM output aligns with the source material provided. The platform cross-references outputs against your source documents using a combination of embedding similarity and fact-checking probes. On SEC filing summarization and Q&A workflows, this approach produces a hallucination rate under 0.3% after verification, compared to 4-8% unverified on the same tasks.

Eval is the ongoing measurement loop: baseline your current hallucination rate during the pilot, track it across workflow changes, and surface regressions before they reach production.

The verification layer

The verification layer is the runtime component that sits between your application and your LLM provider. When your application sends a prompt and receives a response, the verification layer intercepts the response, runs the Fit check against your provided source documents, generates a verification score, and writes the audit log entry.

Your application receives the verified output and the score. If the score falls below a configurable threshold, the platform can suppress the output, return a fallback, or escalate to a human reviewer - depending on how you configure the workflow.

Tamper-evidence

Every audit log entry is hash-chained to the one before it. Any modification to a log entry changes its hash, which breaks the chain and makes the tampering detectable. Log entries are also signed with ECDSA-P256. This structure supports regulatory examination without requiring a separate forensic audit process.

What the platform does not decide

Fact AI Lab does not determine what your LLM is allowed to say. That is your prompt design. The platform measures whether what the LLM said is supported by the source material you provided. Compliance interpretation - whether a given output meets a regulatory standard - remains your legal team's responsibility. The platform gives them the documentation package they need to make that determination.