Core Concepts

Eval-first

Eval-first means checking is part of the build, not an afterthought. In a pilot we first measure how often your current process gets it wrong on one real workflow. Then we check each AI output against your source documents before it reaches a person, and log the result.

This is different from monitoring, which tells you something went wrong after the fact.

The SAFE framework

FactAI Lab works from the SAFE framework: Scope, Audit, Fit, Eval.

Scope defines the boundaries of what the AI is allowed to do in a given workflow: which documents it can use, which kinds of claim are permitted, and what counts as out of scope.

Audit is the record of each AI interaction. We agree with you what each record keeps, for example the prompt, the source passages used, the model and settings, the output and the result of any check.

Fit measures how well an output matches the source material you provided. How that is measured depends on your documents and your task, and we do not publish a general accuracy figure.

Eval is the measurement loop: take a baseline during the pilot, track it as the workflow changes, and surface regressions before they reach your team.

Checks between the AI and your team

For a pilot workflow we can place a check between the model and the person who uses the answer. If a check fails, the workflow can hold the output, use a fallback or send it to a human reviewer. Which of these applies is agreed per workflow.

What we do not decide

FactAI Lab does not determine what your AI is allowed to say. That is your prompt and policy design. Compliance interpretation, whether a given output meets a regulatory standard, remains your legal team's responsibility. We give them the record they need to make that call.