Private-cloud RAG in a single region

Architecture and data flow

Organisation size

20–1000 employees

Data classes

confidential, personal

Constraints

a cloud account you control, with a region that satisfies your residency requirement; a data processing agreement with the provider; no server room, or a workload too seasonal to justify buying hardware

Industries

Professional services, Technology, Financial services, Insurance

Jurisdictions

any

• Cloud GPU instance — 1 × NVIDIA L4 24 GB

  GPU

  NVIDIA L4 24 GB (AWS G6, Azure NVadsA10/NCads equivalents, GCP G2)

  VRAM

  24 GB

  System RAM

  64 GB

  Storage

  1000 GB

  CPU

  8–16 vCPU

  Form factor

  Cloud instance

  Indicative costUS$1 – US$2

  indicative on-demand rental, USD per hour, Aug 2026 — verify against the provider price list for your region

  The private-cloud counterpart of `onprem-small-24gb`: same model sizes, no capital outlay, and a region you choose explicitly. Running it continuously for a year usually costs more than buying the equivalent box, so it suits pilots, bursts and firms without a server room. The cloud provider becomes a data processor — a DPA and a documented region are required.

• Cloud GPU instance — 1–2 × NVIDIA A100 80 GB

  GPU

  NVIDIA A100 80 GB (AWS P4d, Azure NDasr A100 v4, GCP A2)

  VRAM

  80 GB

  System RAM

  256 GB

  Storage

  2000 GB

  CPU

  24–48 vCPU

  Form factor

  Cloud instance

  Indicative costUS$2 – US$12

  indicative on-demand rental, USD per hour, Aug 2026 — verify against the provider price list for your region

  For 32B models with long context, or 70B-class models on two cards. The band covers one to two GPUs and the gap between committed-use and on-demand rates. Check that the GPU family you need exists in the region you are required to stay in before committing to a design.

AWSaws

infrastructure

Azureazure

infrastructure

Dockerdocker

infrastructure

Google Cloudgcp

infrastructure

Identity and SSOidentity-sso

security

Linuxlinux

operations

LLM inferencellm-inference

ml

Networkingnetworking

networking

PostgreSQLpostgresql

data

Retrieval-augmented generationrag

ml

Security hardeningsecurity-hardening

security

Commands are copied from each project’s own documentation, and the page they came from is linked under the step. 0 of 7 steps currently open an evidence record. The rest are linked to their source; §10 says which of those documents were fetched and which were fetched without their anchor being found — two different states, named differently there.

1. 01

   Choose the region, then check the GPU exists in it

   Assessment

   Residency drives the region and the region constrains the hardware: not every GPU family is offered everywhere. Azure documents its GPU families by series — the NCads H100 v5 VMs are "powered by NVIDIA H100 NVL GPU"; AWS G6 instances are "powered by NVIDIA L4 Tensor Core GPUs". Confirm the family, size and quota in your region before the design depends on it.

   Source documentation

2. 02

   Build the network before the application

   Assessment

   One VPC or virtual network, private subnets for the GPU instance and the database, no public IP on either, and access from the office over VPN or a private endpoint. Egress via NAT so the instance can pull images and model weights but nothing can reach it inbound.

3. 03

   Provision managed PostgreSQL and enable pgvector

   Assessment

   Create the managed database in the same region, enable the extension once in the database you will use, and turn on automated backups with a retention window that matches your policy. Keep it private: no public endpoint, no allow-all firewall rule.

   CREATE EXTENSION vector;

   run once in the database that stores embeddings

   Source documentation

4. 04

   Run vLLM on the GPU instanceversion-sensitive

   Assessment

   Use the official image with the GPU attached and the model cache on a persistent disk, so a restart does not re-download tens of gigabytes across the provider network. Set an API key and bind the port to the private subnet only.

   docker run --runtime nvidia --gpus all \
       -v ~/.cache/huggingface:/root/.cache/huggingface \
       --env "HF_TOKEN=$HF_TOKEN" \
       -p 8000:8000 \
       --ipc=host \
       vllm/vllm-openai:latest \
       --model Qwen/Qwen3-0.6B

   from the vLLM docs — substitute the model you sized for

   Source documentation

5. 05

   Run Open WebUI against the managed database

   Assessment

   Point Open WebUI at the managed PostgreSQL and set the vector store to pgvector before loading any documents. Configure OIDC sign-in with your existing identity provider so nobody has a local password in the cloud.

   DATABASE_URL=postgresql://user:password@db-host:5432/openwebui
   VECTOR_DB=pgvector
   PGVECTOR_DB_URL=postgresql://user:password@db-host:5432/openwebui

   Source documentation

6. 06

   Turn on the controls the DPA will ask about

   Assessment

   Encryption at rest with a customer-managed key where the provider offers it, audit logging on the account, retention on the logs, and a documented list of who has administrative access. Record which provider services touch the data and in which region each of them runs.

7. 07

   Put a schedule and a budget alarm on the GPU instance

   Assessment

   A GPU instance left running out of hours is the single largest avoidable cost in this design. Stop it on a schedule if the usage pattern allows, and set a budget alert before the first month rather than after it.

Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.

• The same stack on your own hardware

  If the answer to "can documents leave the building?" is no, or if the GPU would run all day anyway, owning the box is both cheaper over a year and simpler to explain.

  • — No processor in the data path, no DPA needed for the model
  • — Capital spend, a room to put it in, and hardware to maintain

  open-webuivllmpgvector

• A managed model API inside the same tenancy

  Replace the GPU instance with the provider's own model service: no capacity planning, usually a stronger model, and a per-token bill. The retrieval layer stays yours.

  • — No GPU to size, patch or pay for while idle
  • — Prompts and retrieved passages go to a managed service — check its retention and training terms
  • — Model availability differs by region, which can undo the residency design

  azure-openaiaws-bedrockgoogle-vertex-ai

• Self-hostedPrivate company knowledge base (self-hosted RAG)Open WebUI as the employee interface, vLLM serving a Qwen2.5-14B-Instruct model on a single 24 GB GPU, PostgreSQL with pgvector for chats and embeddings, and OIDC single sign-on — all in Docker on one server in your office or colocation rack. Ollama replaces vLLM for teams under about 20 users; a 48 GB GPU lets you run a 32B model for better answers.

Not yet in the ledger

1. Open WebUI — Quick Start (Docker)

   Docs

   not yet fetched

2. Open WebUI — Scaling to multiple users and instances

   Docs

   not yet fetched

3. Open WebUI — SSO (OAuth, OIDC, Trusted Header)

   Docs

   not yet fetched

4. Open WebUI — Hardening

   Docs

   not yet fetched

5. vLLM — Using Docker

   Docs

   not yet fetched

6. vLLM — Quickstart (online serving)

   Docs

   not yet fetched

7. pgvector — README (installation and getting started)

   Readme

   not yet fetched

8. Azure — NCads H100 v5 series GPU virtual machines

   Docs

   not yet fetched

9. AWS — Amazon EC2 G6 instances (NVIDIA L4)

   Vendor page

   not yet fetched

10. Azure — Data, privacy and security for models sold by Azure

    Vendor page

    not yet fetched

11. Google Cloud — Generative AI data governance

    Vendor page

    not yet fetched

12. AWS — Data protection in Amazon Bedrock

    Vendor page

    not yet fetched

Deployed this stack, or hit something this page does not cover? Corrections, sources and implementation reports are what keep a recipe worth reading.

If you would rather not build it, we can introduce a forward-deployed engineer who has deployed this stack before. The enquiry form starts from this recipe.