Method
How an answer is produced
This describes the pipeline as it is built, not as it is planned. Ten steps run in a fixed order; each writes a progress entry before and after it runs, so a run that fails says where it stopped. Every step of every run is stored and can be read back.
01The pipeline
- 01
Reading the questioninterpret
The question becomes a structured deployment brief: objective, jurisdiction, data sensitivity, headcount, technical capability, deployment preference. The brief is the cache key, so two people describing the same problem in different words reach the same run.
- 02
Choosing sourcesplan
Only sources that can answer this brief are queried, out of the registry. A source with no verified domain refuses to run rather than searching the open web under a directory’s name.
- 03
Querying sourcesdiscover
Sources are queried in parallel with a per-source timeout. A source that fails becomes a recorded result with its status, not an exception — an answer built from three of eight sources is still an answer, provided it says so.
- 04
Resolving entitiesnormalise
Retrieved items are resolved against entities we already hold, by canonical domain, then repository name, then alias, then name similarity. Nothing is ever merged automatically; a close match becomes a suggestion for a human.
- 05
Verifying facts against documentsverify
The documents behind the candidate facts are fetched, hashed and stored, and each authored claim’s anchor phrase is located in the text.
- 06
Compliance, hardware, cost and difficultyanalyse
Compliance issue-spotting for the stated jurisdictions, hardware sizing, cost ranges, difficulty and skills — all rules-based, all showing their inputs.
- 07
Scoring the optionsrank
Options are scored on weighted dimensions chosen for this brief. An unknown component is scored conservatively and flagged as unknown rather than passed over.
- 08
Writing the answercompose
The answer document is built from the structured result. With a model configured, three sections’ prose is rewritten and put back through citation enforcement; the model is never the origin of the document.
- 09
Checking every statement against its evidencevalidate
Every statement is checked against the evidence this run holds. See section 04.
- 10
Saving the runpersist
The whole run is stored: the brief, the plan, the queries, the documents, the claims created and reused, the validator report and the answer. It can be explained months later without re-running anything.
The same ledger is shown live while a run executes and stays under the answer as “How this was researched”, with every source query, its status and how long it took.
02Three classes of statement
Every statement in an answer is one of three things, and says which. The label is text, so colour is never the only signal.
FACT
Externally verifiable.
Must point at an evidence record, which points at a document that was fetched, dated and hashed. If the document cannot be retrieved, the claim is stored unverified and renders as “Evidence not verified”.
ASSESSMENT
Our interpretation — difficulty, maturity, risk level, a VRAM estimate, a cost range.
Must state its basis: the formula, the inputs, or the facts it rests on. It is labelled as an assessment wherever it appears.
RECOMMENDATION
A conclusion about your situation, not a general one.
Must name the brief attributes it depends on and at least one fact or assessment, and carries a tier: recommended, strong alternative, consider if, or not recommended because.
03Evidence, anchors and provenance
A fact is three records, not one. The document is what was retrieved — its URL, the hash of its bytes, a second hash of its normalised text, and the time it was fetched. The evidence record is the part of that document that was relied on: the extract, where in the document it sits, and the source’s tier. The claim is the statement itself, with its class, its confidence and its next review date. A claim with no evidence record cannot be rendered as a fact.
The quote stored on an evidence record is text that literally appears in the fetched document. Authored claims declare an anchor phrase, and the sync job searches the document for it. What it finds decides what is stored:
- found
- The anchor phrase was located in the fetched text. The evidence record stores that phrase plus the 240 characters either side of it, and the phrase itself is the source location.
- not_found
- The anchor phrase was given and is not in the text. No evidence record is written, the claim resolves to unverified, its confidence drops, and the page prints “Evidence not verified”. Storing the claim anyway would be the silent gap this system is not allowed to leave.
- not_applicable
- Structured data from an API, where there is no prose to anchor to. The evidence record stores a pointer into the payload instead, such as $.repository.stargazers_count.
Nothing is overwritten in place. A new value supersedes the old one and both stay inspectable, so a figure that changed can be shown to have changed. Every citation carries the document id, the URL, the content hash, the retrieval date and the tier.
04Validation before display
A structural pass runs on every answer, with or without a model:
- 01Every citation is resolved against the evidence records this run actually holds. One that does not resolve is dropped and recorded as a dangling citation.
- 02A statement with at least one surviving citation is marked supported; one with none is marked inferred, and says so on the page.
- 03A fact with no surviving citation is removed from the answer, and the reason is written into the run’s validator report.
With a model configured, a second pass checks entailment per fact, up to a fixed budget per run: the stored quote is put in front of the model with the sentence, and the model is asked whether the quote supports it. A sentence the quote does not support is removed; one it only partly supports is downgraded to inferred and kept. A model error keeps the statement — a model that cannot be reached must not delete a claim that passed the structural check.
One asymmetry is deliberate. A compliance issue that loses its citation is downgraded from fact to assessment rather than dropped. Failing to retrieve an ordinance page is not a reason to stop telling someone the ordinance applies to them.
The counts are published with the answer: how many statements were checked, how many were supported, how many were inferred, what was removed, what was downgraded, and whether the entailment pass ran at all.
05Source priority
Sources are ranked before they are read. The tier travels with every citation, so a reader can weigh a regulator’s page and a directory listing differently without having to know which is which. Tier is an input to a claim’s confidence.
| Tier | Kind of source |
|---|---|
| 1 | legislation / regulator |
| 2 | official vendor legal documentation |
| 3 | official vendor technical documentation |
| 4 | official repository / model card |
| 5 | respected standards organisation |
| 6 | curated specialist directory |
| 7 | general AI directory |
| 8 | reputable editorial source |
| 9 | community discussion |
| 10 | unattributed web content |
A compliance directory is a research prompt, not a conclusion: it identifies an issue, and the issue is then checked against the regulation and the vendor’s own documentation before anything is asserted. Scores published by other services are never reproduced as ours. Every source this deployment can reach is listed with its tier and access method.
06Freshness
Every dynamic fact expires. Each family of fact has a re-check interval; a fact is fresh for the first 60% of it, then ageing, then stale. Past its review date the page prints “Verification may be outdated” and the refresh job re-checks it. Anything unrecognised falls back to the shortest interval — being re-checked too often is cheap, being stale and confident is not.
| Kind of fact | Re-check after |
|---|---|
| Vendor legal, retention and residency documents | 30 days |
| Vendor pricing | 30 days |
| Repository statistics | 7 days |
| Model registry entries | 7 days |
| Regulation status | 90 days |
| Deployment recipe review | 180 days |
An answer prints its oldest retrieval date and the number of citations that are past their review date, so its age is a figure rather than an impression.
07Deterministic mode, and what a model changes
With no language-model key configured the system runs in deterministic mode. That is a supported configuration, not a degraded one. The question is read into a brief by keyword and alias matching over the use-case ontology plus embedding similarity; discovery runs against the official APIs and the seeded registries; compliance, hardware, cost, difficulty and the architecture diagram are rules-based; the answer document is written from templates; and the validator runs its structural pass.
A model changes two things and no others: the brief can be extracted by a model instead of by rules, and three sections’ prose is rewritten and re-checked for citations. A web-search key adds the directory sources, which are otherwise skipped and reported as skipped. No key changes what may be asserted without evidence.
- This deployment
- LLM interpretation: openai
- Web search
- tavily
- Embeddings
- local
08What this does not do
It does not give legal advice. Compliance output is structured issue-spotting against cited primary sources, for you and your counsel to act on.
It does not decide anything for you. A recommendation names the brief attributes it rests on so you can correct them and see the answer change.
It does not claim completeness. A source that failed is named as failed, and an answer built from part of the registry says which part.
It does not fill gaps quietly. Where evidence could not be retrieved, the answer says “Evidence not verified” and raises a warning.