FDE ACADEMY · MODULE 04 · LESSON 04
Security of an AI deployment
The ordinary hardening every system needs, plus the three problems that are specific here: over-broad retrieval, prompt leakage, and where your model weights came from.
- Effort
- Half a day, including a deliberate attempt to retrieve something you should not.
- Skills
- security-hardening · identity-sso · rag
Free and self-paced. No certification is offered.
Before you startYou have single sign-on and a reverse proxy from module 03.
01Most of it is not new
Patching, least privilege, secrets outside the image, encrypted storage, restricted networks, backups you have restored from, and an incident path with a name and a number on it. None of this is specific to AI, all of it applies, and skipping it because the project is an AI project is the most common way these deployments fail a review.
Do the ordinary work first, and do it in the order the recipe puts it: hardening before the first login, not after the pilot.
02Three problems that are specific
Over-broad retrieval is the big one. An index built without permissions will answer from any document it holds, so the access control you configured at the application layer can be bypassed by asking a question. Test it as an attack, not as a configuration review.
Prompt leakage is the second: instructions, examples, retrieved passages and previous turns all sit in one context, and content in a retrieved document can influence the model’s behaviour. Treat retrieved text as untrusted input, keep anything sensitive out of system prompts, and do not give a model tools whose misuse you have not considered.
Model provenance is the third and least discussed. Weights are large binaries downloaded from a registry. Record which model, which revision and which source, verify what you can, and pin it — so the artefact running next year is the one you evaluated this year.
- Test retrieval permissions by asking questions, not by reading a permissions table.
- Treat every retrieved passage as untrusted content that a model will read as if it were instruction.
- Pin model versions and record where the weights came from, as you would for any other dependency.
03What the customer’s reviewer will ask
Who can reach it, who can sign in, what they can see, what is logged, what is retained, what happens when someone leaves, what happens when it breaks, and who is called. If your deployment can answer all eight in writing, the review is short.
The artefacts from module 03 answer most of them. This lesson adds the ninth: what you tried to break, and what happened.
04Do this
PRACTICAL TASK
Harden the deployment, then try to get at something you should not
Complete the recipe’s hardening step, then run a deliberate misuse session against your own deployment and write up what you found.
What you need
- Your deployment with two accounts at different access levels
- One document only the higher-privileged account should be able to read
Steps
- 01
Work through the recipe’s hardening step in full, before any real user is invited in.
- 02
As the lower-privileged account, try ten different questions designed to make the system quote the restricted document. Record every response.
- 03
Put a document into the collection containing text that instructs the assistant to ignore its previous instructions, and see what happens when it is retrieved.
- 04
Record which model and revision you are running and where the weights came from, then pin it.
- 05
Read the security and confidentiality compliance notes on the recipe and check each of your findings against them.
You are done when
- Ten attempts to reach the restricted document, with the outcome of each written down.
- You know what your deployment does with instruction-like text inside a retrieved document.
- The model and revision are pinned and recorded, with the source they came from.
05Where these facts live
This lesson does not restate anything that is already recorded with its evidence elsewhere on the site. These are the pages it leans on.
- Private company knowledge base
Hardening is step eight, before the first collection is loaded and before anyone logs in.
- Contract review pipeline
A higher-sensitivity deployment whose compliance notes are worth reading against your own.
- A controlled AI learning environment
A deployment whose users are children: the controls that follows from that, stated as design decisions.
Ticks are stored in your browser only. They are not sent anywhere, they are not attached to an account, and clearing your browser data removes them.