Skip to content
Is there an AI for this?

Method

How an answer is produced

This describes the pipeline as it is built, not as it is planned. Ten steps run in a fixed order; each writes a progress entry before and after it runs, so a run that fails says where it stopped. Every step of every run is stored and can be read back.


01The pipeline

  1. 01

    Reading the questioninterpret

    The question becomes a structured deployment brief: objective, jurisdiction, data sensitivity, headcount, technical capability, deployment preference. The brief is the cache key, so two people describing the same problem in different words reach the same run.

  2. 02

    Choosing sourcesplan

    Only sources that can answer this brief are queried, out of the registry. A source with no verified domain refuses to run rather than searching the open web under a directory’s name.

  3. 03

    Querying sourcesdiscover

    Sources are queried in parallel with a per-source timeout. A source that fails becomes a recorded result with its status, not an exception — an answer built from three of eight sources is still an answer, provided it says so.

  4. 04

    Resolving entitiesnormalise

    Retrieved items are resolved against entities we already hold, by canonical domain, then repository name, then alias, then name similarity. Nothing is ever merged automatically; a close match becomes a suggestion for a human.

  5. 05

    Verifying facts against documentsverify

    The documents behind the candidate facts are fetched, hashed and stored, and each authored claim’s anchor phrase is located in the text.

  6. 06

    Compliance, hardware, cost and difficultyanalyse

    Compliance issue-spotting for the stated jurisdictions, hardware sizing, cost ranges, difficulty and skills — all rules-based, all showing their inputs.

  7. 07

    Scoring the optionsrank

    Options are scored on weighted dimensions chosen for this brief. An unknown component is scored conservatively and flagged as unknown rather than passed over.

  8. 08

    Writing the answercompose

    The answer document is built from the structured result. With a model configured, three sections’ prose is rewritten and put back through citation enforcement; the model is never the origin of the document.

  9. 09

    Checking every statement against its evidencevalidate

    Every statement is checked against the evidence this run holds. See section 04.

  10. 10

    Saving the runpersist

    The whole run is stored: the brief, the plan, the queries, the documents, the claims created and reused, the validator report and the answer. It can be explained months later without re-running anything.

The same ledger is shown live while a run executes and stays under the answer as “How this was researched”, with every source query, its status and how long it took.


02Three classes of statement

Every statement in an answer is one of three things, and says which. The label is text, so colour is never the only signal.

  • FACT

    Externally verifiable.

    Must point at an evidence record, which points at a document that was fetched, dated and hashed. If the document cannot be retrieved, the claim is stored unverified and renders as “Evidence not verified”.

  • ASSESSMENT

    Our interpretation — difficulty, maturity, risk level, a VRAM estimate, a cost range.

    Must state its basis: the formula, the inputs, or the facts it rests on. It is labelled as an assessment wherever it appears.

  • RECOMMENDATION

    A conclusion about your situation, not a general one.

    Must name the brief attributes it depends on and at least one fact or assessment, and carries a tier: recommended, strong alternative, consider if, or not recommended because.


03Evidence, anchors and provenance

A fact is three records, not one. The document is what was retrieved — its URL, the hash of its bytes, a second hash of its normalised text, and the time it was fetched. The evidence record is the part of that document that was relied on: the extract, where in the document it sits, and the source’s tier. The claim is the statement itself, with its class, its confidence and its next review date. A claim with no evidence record cannot be rendered as a fact.

The quote stored on an evidence record is text that literally appears in the fetched document. Authored claims declare an anchor phrase, and the sync job searches the document for it. What it finds decides what is stored:

found
The anchor phrase was located in the fetched text. The evidence record stores that phrase plus the 240 characters either side of it, and the phrase itself is the source location.
not_found
The anchor phrase was given and is not in the text. No evidence record is written, the claim resolves to unverified, its confidence drops, and the page prints “Evidence not verified”. Storing the claim anyway would be the silent gap this system is not allowed to leave.
not_applicable
Structured data from an API, where there is no prose to anchor to. The evidence record stores a pointer into the payload instead, such as $.repository.stargazers_count.

Nothing is overwritten in place. A new value supersedes the old one and both stay inspectable, so a figure that changed can be shown to have changed. Every citation carries the document id, the URL, the content hash, the retrieval date and the tier.


04Validation before display

A structural pass runs on every answer, with or without a model:

  1. 01Every citation is resolved against the evidence records this run actually holds. One that does not resolve is dropped and recorded as a dangling citation.
  2. 02A statement with at least one surviving citation is marked supported; one with none is marked inferred, and says so on the page.
  3. 03A fact with no surviving citation is removed from the answer, and the reason is written into the run’s validator report.

With a model configured, a second pass checks entailment per fact, up to a fixed budget per run: the stored quote is put in front of the model with the sentence, and the model is asked whether the quote supports it. A sentence the quote does not support is removed; one it only partly supports is downgraded to inferred and kept. A model error keeps the statement — a model that cannot be reached must not delete a claim that passed the structural check.

One asymmetry is deliberate. A compliance issue that loses its citation is downgraded from fact to assessment rather than dropped. Failing to retrieve an ordinance page is not a reason to stop telling someone the ordinance applies to them.

The counts are published with the answer: how many statements were checked, how many were supported, how many were inferred, what was removed, what was downgraded, and whether the entailment pass ran at all.


05Source priority

Sources are ranked before they are read. The tier travels with every citation, so a reader can weigh a regulator’s page and a directory listing differently without having to know which is which. Tier is an input to a claim’s confidence.

TierKind of source
1legislation / regulator
2official vendor legal documentation
3official vendor technical documentation
4official repository / model card
5respected standards organisation
6curated specialist directory
7general AI directory
8reputable editorial source
9community discussion
10unattributed web content

A compliance directory is a research prompt, not a conclusion: it identifies an issue, and the issue is then checked against the regulation and the vendor’s own documentation before anything is asserted. Scores published by other services are never reproduced as ours. Every source this deployment can reach is listed with its tier and access method.


06Freshness

Every dynamic fact expires. Each family of fact has a re-check interval; a fact is fresh for the first 60% of it, then ageing, then stale. Past its review date the page prints “Verification may be outdated” and the refresh job re-checks it. Anything unrecognised falls back to the shortest interval — being re-checked too often is cheap, being stale and confident is not.

Kind of factRe-check after
Vendor legal, retention and residency documents30 days
Vendor pricing30 days
Repository statistics7 days
Model registry entries7 days
Regulation status90 days
Deployment recipe review180 days

An answer prints its oldest retrieval date and the number of citations that are past their review date, so its age is a figure rather than an impression.


07Deterministic mode, and what a model changes

With no language-model key configured the system runs in deterministic mode. That is a supported configuration, not a degraded one. The question is read into a brief by keyword and alias matching over the use-case ontology plus embedding similarity; discovery runs against the official APIs and the seeded registries; compliance, hardware, cost, difficulty and the architecture diagram are rules-based; the answer document is written from templates; and the validator runs its structural pass.

A model changes two things and no others: the brief can be extracted by a model instead of by rules, and three sections’ prose is rewritten and re-checked for citations. A web-search key adds the directory sources, which are otherwise skipped and reported as skipped. No key changes what may be asserted without evidence.

This deployment
LLM interpretation: openai
Web search
tavily
Embeddings
local

08What this does not do

  • It does not give legal advice. Compliance output is structured issue-spotting against cited primary sources, for you and your counsel to act on.

  • It does not decide anything for you. A recommendation names the brief attributes it rests on so you can correct them and see the answer change.

  • It does not claim completeness. A source that failed is named as failed, and an answer built from part of the registry says which part.

  • It does not fill gaps quietly. Where evidence could not be retrieved, the answer says “Evidence not verified” and raises a warning.