Logging, monitoring, and the routine that uses them

You need to know who asked, when, which documents were retrieved, how long it took and whether it failed. You usually do not need to store the text of the question, and once you do, your logs contain whatever your users typed — which may be personal data, client material, or both, sitting in a system with different retention and access rules from the documents themselves.

Decide this deliberately, write it down, and set a retention period. Prompt content is sometimes genuinely necessary for evaluation; when it is, it should be a decision with a period attached rather than a default nobody chose.

Latency at the ninety-fifth percentile, because averages hide the experience people complain about. Error rate by kind, separating model errors from ingestion errors. Queue depth or concurrency, so you see the capacity limit from lesson 01 approaching. GPU memory, because the cache term grows with use. And a retrieval quality figure re-run on the same twenty questions from module 02.

The last one is the only one that tells you whether the deployment is still good. The first four tell you whether it is still up, and a system can be up and useless.

Telemetry nobody looks at is storage. What you leave behind is a routine: a named owner, a weekly half-hour, a short list of things to check, and a decision about what triggers action. The flagship recipe ends with exactly that step, and it is the one most likely to be skipped because nothing breaks the day you skip it.

Include a review date. Documentation, model choices and regulatory guidance all age, and this site marks its own claims as stale past their review window for the same reason.