Reference deployment · R-01
Private internal document search for a Hong Kong professional-services firm
Forty fee earners, twenty years of client files on a shared drive, and an obligation not to disclose any of it. A self-hosted retrieval stack on one GPU server, with the firm’s own identity provider deciding who can see which matter.
- Source
- Designed by us against a stated scenario. 0 of 16 compliance issues are anchored in a fetched instrument.
- Evidence
- none on this page — it links to the pages that hold it
| Parameter | Declared value |
|---|---|
| Scenario | Hong Kong professional-services firm, 40 fee earners plus 15 support staff |
| Industry | Professional services |
| Organisation size | 55 employees — Medium (50–249) |
| People using the system | 40 |
| Technical capability | IT team |
| Jurisdictions | Hong Kong |
| Data residency required | Hong Kong |
| Data types declared | client documents, legal documents, contracts, internal documents, personal data |
| Languages | English, Traditional Chinese |
| Delivery assessed | self hosted |
01Problem
Fee earners cannot find precedent. The firm holds roughly twenty years of matter files on a shared drive, organised by client and year, and the only reliable way to find a clause somebody drafted in 2019 is to ask the person who drafted it. That person is often the bottleneck, and sometimes has left.
The firm has tried desktop search. It matches filenames and exact strings, which is the wrong shape for the question being asked — "have we argued this before, and how did it go" is a question about meaning, and the material is a mixture of scanned PDFs, Word files and email attachments in two languages.
What is at stake is not efficiency in the abstract. Every hour spent re-deriving an argument the firm has already made is an hour billed to a client for work already done once.
02Constraints
- 01Client material must not leave the firm’s networkhard — eliminates optionsThe corpus is client work product held under professional obligations of confidentiality. The firm’s position is that no third party processes it, and that this is an architectural property rather than a contractual one.
- 02Permissions must mirror the matter walls that already existhard — eliminates optionsThe firm runs information barriers between certain matters. A search system that flattens them is worse than no search system, because it makes a breach fast.
- 03Two languages, mixed within documentshard — eliminates optionsEnglish and Traditional Chinese, frequently in the same file. Retrieval and the model both have to handle this without a separate pipeline per language.
- 04One part-time IT person, no engineering teamshapingWhatever is deployed has to be operable by an IT generalist with a runbook, and upgradeable without a specialist.
- 05Scanned material is a large minority of the corpusshapingOlder matters are scans. Text extraction quality decides retrieval quality for anything before roughly 2015.
03Recommended solution
Self-hosted retrieval on one GPU server, behind the firm’s existing identity provider
The hard constraint decides the class before any tool is compared: if client material may not be processed by a third party, the system runs on hardware the firm controls, and the only remaining questions are which components and how large the server has to be.
The permission constraint decides the sequence. Retrieval is added after the assistant is in use, and the first corpus is a single practice group whose matter walls are simple — because the index inherits whatever access control the source has, and a shared drive’s access control is usually a project in itself.
The language constraint decides the model and the embedding: both have to handle Traditional Chinese in the same corpus as English, which rules out several otherwise reasonable English-first choices.
Component
Why — assessment
- 01InterfaceOpen WebUIDocumented OIDC sign-in against the firm’s existing provider, and knowledge collections that can be scoped per group rather than per user.
- 02IdentityThe firm’s existing OIDC provider (Keycloak where there is none)Matter walls already exist in the directory. Re-expressing them in a second user store is how the two come to disagree.
- 03IngestionDoclingConverts the PDF and Office material, including scans, into text with structure preserved — headings survive, which is what makes chunking by section possible.
- 04Vector storePostgreSQL with pgvectorThe firm already runs PostgreSQL for another system, so this adds an extension rather than a database to back up, patch and monitor.
- 05InferencevLLMBatching across concurrent sessions is what lets one GPU serve forty people at 09:15; a single-session server would need several.
- 06ModelAn open-weights instruct model with Chinese and English coverage, 4-bitThe corpus is bilingual within documents. Quantisation is what fits a capable model on a single 48 GB card alongside the KV cache for forty sessions.
04Architecture
External data transfer
External data transfer: NONE
No edge in this design crosses the boundary drawn below. Boundary: COMPANY NETWORK.
Components
- Fee earners and support staff — people
- Document ingestion (Docling) — storage
- Reverse proxy (TLS, internal network only) — network
- Open WebUI — application
- Firm OIDC provider (matter groups) — identity
- Retrieval layer (chunk by section, filter by group) — retrieval
- PostgreSQL + pgvector (embeddings) — vector store
- PostgreSQL (chats, users, permissions) — database
- Matter files (existing shared storage, read-only mount) — storage
- vLLM (OpenAI-compatible server) — inference server
- Open-weights bilingual instruct model (4-bit) — model
- Firm GPU server (48 GB) — hardware
Connections
- Fee earners and support staff to Reverse proxy (TLS, internal network only) — HTTPS (confidential data)
- Reverse proxy (TLS, internal network only) to Open WebUI — reverse proxy (confidential data)
- Open WebUI to Firm OIDC provider (matter groups) — OIDC sign-in (personal data)
- Document ingestion (Docling) to Retrieval layer (chunk by section, filter by group) — documents to index (confidential data)
- Open WebUI to Retrieval layer (chunk by section, filter by group) — question + user groups (confidential data)
- Retrieval layer (chunk by section, filter by group) to PostgreSQL + pgvector (embeddings) — embedding search (confidential data)
- Retrieval layer (chunk by section, filter by group) to PostgreSQL (chats, users, permissions) — documents + permissions (confidential data)
- Open WebUI to PostgreSQL (chats, users, permissions) — chats, users, settings (personal data)
- Retrieval layer (chunk by section, filter by group) to Matter files (existing shared storage, read-only mount) — original files (confidential data)
- Retrieval layer (chunk by section, filter by group) to vLLM (OpenAI-compatible server) — prompt + retrieved passages (confidential data)
- vLLM (OpenAI-compatible server) to Open-weights bilingual instruct model (4-bit) — loaded weights
- Open-weights bilingual instruct model (4-bit) to Firm GPU server (48 GB) — GPU memory
- vLLM (OpenAI-compatible server) to Firm GPU server (48 GB) — GPU memory
External data transfer · NONE
- No edge in this design crosses out of the company network.
- Model weights are downloaded once at setup. After that the server answers questions with no outbound connection.
- Container image pulls and operating-system patching are the only remaining outbound flows, and they carry no client material.
05Compliance
Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.
16 of 27 rules fired for this scenario, across Hong Kong.
Cross Cutting
Self-hosting moves the security obligation to you
Keeping data on your own hardware answers the transfer question and creates an operations question. Patching, backups, key management, monitoring and incident response are now yours, and an unpatched inference server on the office network is a worse outcome than a well-run vendor.
- Required checks
- Name the person responsible for patching each component, and the cadence.
- Confirm backups exist, are encrypted, and have been restored at least once.
- Confirm there is an incident response path that includes this system.
- Technical controls
- Encrypt at rest and in transit, including between the application and the inference server.
- Centralise authentication through the existing identity provider rather than local accounts.
- Keep an audit log of who queried what, and protect it from the people it records.
- Subscribe to security advisories for each component and track upgrade lag.
Being able to reconstruct a decision months later
The question that arrives after a complaint is what the system was shown and what it produced on a particular day. Models change, prompts change, and indexes are rebuilt, so the answer has to be recorded at the time. Without it, the only available response is that the output cannot be reproduced.
- Required checks
- Decide what is recorded per interaction: model and version, prompt template version, retrieved document ids, output, reviewer and outcome.
- Set how long those records are kept, balanced against the retention duties that also apply to them.
- Vendor questions
- Does the vendor pin model versions, and how much notice is given before a model is retired or changed?
- Technical controls
- Version prompt templates in source control and log the version used.
- Log the model identifier and version returned by the provider, not the one you requested.
Confidentiality duties bind independently of data protection law
Material can be entirely free of personal data and still be the material a contract stops you disclosing. Client retainers, non-disclosure agreements, supplier contracts and common-law duties are the usual sources, and several of them require consent before a third party processes the material at all — which a model API call is.
- Required checks
- Review the confidentiality clauses in the contracts covering the material going into the system.
- Identify any contract requiring notice or consent before a subcontractor processes the material.
- Decide whether the deployment needs a confidentiality carve-out negotiated into new contracts.
- Vendor questions
- Will the vendor accept a confidentiality undertaking beyond its standard terms?
- Which staff at the vendor can access customer content, under what controls?
- Technical controls
- Segregate the most sensitive corpora into an index that the general assistant cannot reach.
Human review required — take this to your counsel
An AI deployment creates new copies of the data
Vector indexes, prompt logs, completion caches, evaluation datasets, fine-tuning checkpoints and backups are all copies of the source material in places the existing retention schedule does not mention. Deletion requests are the moment this is discovered, because deleting the source document does not delete its embedding.
- Required checks
- List every store the deployment creates and add each to the retention schedule.
- Establish how a deletion request propagates to the index, the caches and the logs.
- Establish how long backups keep material that has been deleted from the live system.
- Vendor questions
- What does the vendor retain, where, and for how long after we delete our copy?
- Technical controls
- Store the source document id with every embedding so deletion can cascade.
- Set time-to-live on prompt and completion logs rather than relying on manual cleanup.
Who can reach the model, the index and the weights
A self-hosted stack has three access surfaces that are easy to leave open: the inference endpoint, the vector index, and the weights on disk. Retrieval also carries an authorisation problem an ordinary application does not have — the index must not return a document to someone who could not open it in the source system.
- Required checks
- Confirm the inference endpoint is not reachable from outside the network and requires authentication.
- Confirm retrieval filters by the requesting user’s permissions, not only by relevance.
- Confirm who can read the model files and the index volume at the operating-system level.
- Technical controls
- Bind the inference server to a private interface and put an authenticating proxy in front of it.
- Carry document-level access control into the index and enforce it at query time.
- Encrypt the volume holding the weights and the index, and restrict it to the service account.
- Rotate API keys and keep them out of client-side code and container images.
What ends up in a prompt, and where it goes next
Even with inference inside the network, prompts and retrieved context accumulate in logs, traces and caches, and system prompts can often be extracted from the output. The leak path is internal rather than external, but it is still a copy of the source material in a new place.
- Required checks
- Write down which categories of information may be entered into a prompt, and tell people.
- Establish what the system prompt contains and whether disclosing it would matter.
- Establish which shadow tools staff are already using; the policy has to name the permitted ones.
- Vendor questions
- Are prompts and completions retained, for how long, and can retention be set to zero?
- Are prompts used for abuse monitoring, and if so who can read them and for how long?
- Technical controls
- Redact or block high-risk patterns before the prompt leaves the application.
- Keep prompt and completion logs out of general-purpose observability tools.
- Set an explicit retention period on prompt logs and enforce it.
Rights in the material you index and in what the model produces
Two separate questions get conflated. First, whether you hold the rights to index, embed or fine-tune on the material you feed the system — licensed databases, customer documents and third-party reports often say you may not. Second, whether generated output that closely reproduces protected work creates exposure. Both are contract and copyright questions that vary by jurisdiction, and neither is answered by the model licence.
- Required checks
- List the corpora going into the index and confirm each one’s terms permit machine processing and storage of derived embeddings.
- Check whether any source is a licensed third-party database with an anti-scraping or anti-derivative clause.
- Decide, and write down, who owns the output in your customer contracts.
- Vendor questions
- Does the provider offer any indemnity for third-party intellectual-property claims arising from output, and what conditions attach to it?
- Technical controls
- Record the source document for every retrieved passage so an output can be traced to what it was grounded on.
Hong Kong
DPP4 — all practicable steps to protect the data
DPP4 requires all practicable steps to protect personal data against unauthorised or accidental access, processing, erasure, loss or use, including where a data processor holds it. For a retrieval system that normally means access control that mirrors the source repository, encryption at rest and in transit, and an audit log — and it is the principle the Commissioner reaches for after a breach.
- Required checks
- Confirm retrieval cannot return a document to a user who could not open it at source.
- Confirm encryption at rest for the index, the logs and any model artefacts.
- Confirm there is a tested route to detect and respond to unauthorised access.
- Vendor questions
- What security certifications do you hold, and can we see the current report rather than a badge?
- Which of your staff can access customer content, and under what controls?
- Technical controls
- Enforce document-level permissions at query time, not only at ingestion time.
- Authenticate through the existing identity provider rather than local accounts.
- Keep an access log covering who queried what, retained separately from the application.
DPP3 — a new purpose needs express and voluntary consent
DPP3 prohibits using personal data for a new purpose that is not, or is unrelated to, the purpose for which it was collected, unless the data subject gives express and voluntary consent. This is the provision that decides whether existing client and staff records can be used to power an assistant at all, and it is answered by reading what people were told at collection.
- Required checks
- Retrieve the personal information collection statements actually used for the data in scope.
- Assess whether the AI use is the original purpose, a directly related purpose, or a new one.
- Where it is a new purpose, plan how express consent is obtained — or narrow the corpus.
- Technical controls
- Tag indexed documents with the collection basis so an out-of-scope source is visible.
Human review required — take this to your counsel
PCPD Model Framework — AI governance and the level of human oversight
The PCPD’s Artificial Intelligence: Model Personal Data Protection Framework recommends that organisations formulate policies, practices and procedures when they procure, implement and use AI solutions, taking into account its recommended measures in AI strategy and governance, risk assessment and human oversight, customisation of AI models and implementation and management of AI systems, and communication and engagement with stakeholders. It is guidance rather than law.
- Required checks
- Decide who owns AI governance internally and what they sign off before a system goes live.
- Run a risk assessment that sets the level of human oversight for this system, and record it.
- Train the people who will use the system, and record that you did.
- Vendor questions
- What documentation do you provide to support a customer’s own AI risk assessment?
- Technical controls
- Give reviewers the retrieved sources alongside the output so oversight is possible in practice.
DPP1 — collection must be lawful, necessary and not excessive
DPP1 allows personal data to be collected only for a lawful purpose directly related to a function or activity of the data user, and requires the data collected to be necessary and adequate but not excessive for that purpose. Indexing a whole document store for an assistant is a collection decision made at scale, and "we indexed everything because it was easier" is the shape of an excessive-collection finding.
- Required checks
- State the purpose the AI system serves, in the terms the business would defend it in.
- List which repositories are indexed and confirm each is necessary for that purpose.
- Check whether the personal information collection statement given to staff and clients covers this use.
- Technical controls
- Scope the index by folder, label or classification rather than by whole drive.
- Exclude repositories that exist for a different purpose, such as HR files, unless they are in scope deliberately.
PCPD checklist — write the internal generative AI policy
The PCPD’s 2025 checklist is explicit about what an internal policy should cover: which generative AI tools are permitted, the permissible purposes, clear instructions on the types and amounts of information that can be inputted into the Gen AI tools, lawful and ethical use and bias, which devices and which categories of employee may use the tools, AI incident reporting, and the consequences of a violation.
- Required checks
- Name the permitted tools, including whether publicly available tools are allowed at all.
- Say which categories of employee may use them, on which devices.
- Give concrete instructions on what may go into a prompt, with examples rather than principles.
- Add AI incidents to the existing incident response plan and say what an AI incident is.
- Technical controls
- Make the sanctioned tool easier to reach than the unsanctioned one; policy alone does not move behaviour.
DPP2 — accuracy, and no longer than necessary
DPP2 requires all practicable steps to keep personal data accurate and not to keep it longer than is necessary for the purpose it is used for. Where a data processor is engaged, the data user must adopt contractual or other means to ensure the processor complies with the same retention requirement. An AI deployment multiplies the copies: embeddings, prompt logs, cached completions and evaluation sets all fall inside this.
- Required checks
- Add every store the deployment creates to the retention schedule with a stated period.
- Confirm the contract with any processor caps their retention and that you can verify it.
- Decide what "accurate" means for a generated answer, and how a wrong one is corrected at source.
- Vendor questions
- How long do you retain prompts, outputs and logs, and can that be reduced contractually?
- Technical controls
- Set a time-to-live on prompt and completion logs.
- Re-index on a schedule so the assistant does not answer from a superseded document.
DPP6 — access and correction, and the log book
DPP6 gives data subjects the right to request access to and correction of their own personal data, with detailed provisions in Part 5 of the Ordinance on manner and timeframe, on when a request may be refused, and on maintaining a log book recording every refusal. A vector index and a prompt log are places that data now lives, and a data access request reaches them.
- Required checks
- Establish how a data access request is answered when the data also sits in an index or a log.
- Establish how a correction propagates from the source document to the index.
- Confirm the refusal log book exists and covers requests touching the AI system.
- Vendor questions
- How do we extract or delete a specific individual’s data from your service, and how long does it take?
- Technical controls
- Keep the source document id on every embedding so a subject can be located across stores.
DPP5 — openness about policies, practices and what you hold
DPP5 requires all practicable steps to make a data user’s personal data policies and practices open, along with the kind of personal data held and the main purposes for holding it. Deploying an assistant over personal data changes the practices, so the published statement has to change with it.
- Required checks
- Update the privacy policy and internal data handling notice to describe the AI processing.
- Say plainly, where people will see it, that an AI system is involved and what it does.
Data stays inside — but the weights still came from outside
Keeping documents on your own server removes the transfer question for the content. It does not remove the supply chain: model weights, container images and dependencies are downloaded from third parties, and a gated repository records who downloaded what and when. That is a procurement and integrity matter rather than a personal data transfer, and it is worth stating explicitly so nobody assumes the system is hermetic.
- Required checks
- Record which model revision you downloaded, from where, and under which licence.
- Note that accepting gated access terms is a contract, and check who accepted it.
- Technical controls
- Verify checksums on downloaded weights and pin the revision rather than tracking a moving tag.
- Mirror the artefacts internally so a rebuild does not depend on the upstream still being there.
Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.
What this reading does not know
- Whether any of the data falls into a special or sensitive category.
- Whether any material is covered by legal professional privilege.
06Cost range
| Line | One-off |
|---|---|
| Software | HK$0 |
| Model | HK$0 |
| Infrastructure | HK$93,000 – HK$170,000 |
| Implementation | HK$72,000 – HK$390,000 |
| Total | HK$170,000 – HK$560,000 |
| From year two | HK$81,000 – HK$400,000 a year |
Assumptions
- Software: nil. Every component in this recipe is open source and self-installed, so there is no licence fee. Check each licence for use limits before relying on that (see the recipe's open-source-licensing compliance note).
- Model: nil per month. Open weights are downloaded once and served on your own hardware, so there is no per-token charge; the cost of running them sits in infrastructure.
- Infrastructure: On-premise single 48 GB GPU server, a one-off purchase of 12000–22000 USD (indicative build cost for the complete machine, USD, Aug 2026 — verify with a local supplier). Typically written down over three to five years.
- The purchase price excludes electricity, cooling, UPS, rack space, offsite backup and hardware support cover; those are bracketed in the from-year-two line instead of being left as an unpriced list.
- Implementation: 12–26 FDE-days at 6000–15000 HKD per day. One-off. Covers installation, configuration, ingestion of an initial corpus, an evaluation pass and handover; excludes ongoing support and internal staff time.
- Infrastructure converted from USD using HKMA Linked Exchange Rate System: HK$7.75–7.85 to one US dollar.
- Basis: one-off only — implementation and the hardware purchase. This recommendation has no recurring monthly line at all, so the total is what it costs to stand up, not a monthly bill. What it costs to keep running is the from-year-two line below.
- Sized for 40 users. Every figure is an assessment from the inputs listed above, not a quotation.
- From year two this costs roughly HKD 81,000–HKD 400,000 a year to keep running. That is an order of magnitude for a budget conversation, not a quotation: every line below is derived from the hardware price and the FDE day rate already shown, and no supplier was asked.
- From year two — running the hardware: HKD 9,300–HKD 43,000 a year, taken as 10–25% of the HKD 93,000–HKD 170,000 purchase price and covering electricity, cooling or rack space, offsite backup and hardware support cover together. That percentage band is our own bracket, not a supplier quote — no price was fetched for any of those four items. Replace it with your electricity tariff, rack charge and a support quote.
- From year two — ongoing time: 12–24 FDE-days a year (one to two days a month, the figure the recipe's own cost model uses for support and patching) at HKD 6,000–HKD 15,000 per day = HKD 72,000–HKD 360,000. Model updates, index rebuilds, OS and driver patching, and onboarding questions. Internal staff time costs the same hours whether or not it is invoiced.
- Rounding: every figure above is rounded to two significant figures, because each is an indicative band rather than a quotation. The rounded lines may therefore not add up exactly to the rounded total.
Produced by the same cost model an answer uses, from the declared user count, the hardware profile and the implementation band below. Every figure is a range rounded to two significant figures, and the assumptions above are the model’s own.
07Difficulty
2 / 5
A few days, mostly configuration
- AssessmentStarting point 2/5: software you install, run and keep running on your own machines.
- Assessment+0.5 You run the model server yourself: GPU drivers, quantisation choice, memory headroom and restarts are all yours to own.
- Assessment+0.5 Single sign-on adds an identity provider, group-to-role mapping and a joiner/leaver process to the deployment.
- Assessment-0.5 An in-house IT team can run the server, the backups and the identity provider already.
- AssessmentAssessed difficulty 2/5 (2 +0.5 +0.5 -0.5 = 2.5, rounded and capped to 1–5).
08Implementation path
This reference implements the Private company knowledge base (self-hosted RAG) recipe, which carries the components, the ordered steps, the commands and the evidence behind each one.
- Implementation
- 12–26 FDE-days
- Use case
- Internal company search
AssessmentAbove the recipe’s own 8–20 day band: this scenario adds bilingual ingestion of scanned material and a matter-group permission mapping, neither of which the base recipe carries.
This reference would not suit
- A firm whose material may be processed by a vendor under a data-processing agreement — the hard constraint is what makes this expensive, and without it a hosted product is a shorter path.
- A corpus that lives in five systems with inconsistent permissions. This design assumes one authoritative store and one directory; connector work would change both the effort and the shape.
- Fewer than about fifteen users. Below that the GPU server is idle most of the day and the arithmetic favours renting capacity in a single region instead.
09Evidence
No evidence record for this design could be resolved when this page was rendered. The documents it rests on are named on the recipe page; the compliance statements above are shown as assessments rather than facts because of it.
8 of 8 declared references had no stored evidence record at render time and are not listed above.
10Hire or build — next step
If your situation resembles the scenario at the top of this page, the next step is either to build it from the recipe or to describe your own constraints and have them read against the rules rather than against this one.
Or ask about your own case