Use case
Private LLM
Using a language model without your data leaving a boundary you control — a country, a cloud region, or your own network. The constraint is where processing happens and who can see the prompts, not which model runs.
- Source
- Editorial ontology entry — no fetched document behind this page
- Evidence
- none on this page — it links to the pages that hold it
- Category
- Infrastructure
- Typical risk
- high
- Entry
- editorial · reviewed 25 Aug 2026
01What this is
A private LLM deployment is defined by a boundary: data must stay within a jurisdiction, a tenancy, or a network. That boundary is met in three ways — a hosted model in a chosen region with contractual controls, a single-tenant deployment in your own cloud account, or self-hosted inference on your own hardware.
A good deployment writes the boundary down first, then verifies it against what the vendor actually publishes: processing regions, subprocessors, retention and training terms, and whether logs and abuse-monitoring copies leave the region. Self-hosting removes the vendor question but adds operations, capacity and patching.
Pitfalls: assuming a regional endpoint means regional processing for every subsystem; overlooking logging and support access; and treating "no training on customer data" as equivalent to "no data leaves". Ask what leaves, where it goes, how long it stays and who may read it — in that order.
- Typical data
- client documents, personal data, financial records, internal documents
- Solution classes it admits
- Private cloud, Self-hosted
02Deployment options
- ASSESSMENT
On a neutral reading of this use case, Self-hosted is a strong alternative, Private cloud is a strong alternative and Enterprise SaaS is conditional.
- ASSESSMENT
Read as a generic reading of this page, not a recommendation: no organisation, size, jurisdiction, budget or technical capability has been supplied, so wherever an option depends on one of those, it says "unknown". Ask your own question to get a verdict that accounts for them.
Private cloud
STRONG ALTERNATIVEManaged model API or private model deployment in your cloud account
- ASSESSMENT
Two patterns fit inside one account in a region you name: call a managed foundation-model API such as Amazon Bedrock, Microsoft Foundry / Azure OpenAI, or Vertex AI; or deploy an open-weight model on GPU compute you control. Your application, retrieval layer, storage, identity and logs remain in your cloud boundary in both patterns.
- ASSESSMENT
The managed-API pattern can use closed-source frontier models without buying or operating GPUs. It is usually the fastest way to build a custom workflow, but prompts and retrieved context are processed by the managed service, so model availability, retention, abuse monitoring and regional routing must be checked for the exact feature and endpoint.
- ASSESSMENT
The private-model pattern gives more control over weights, serving and network paths, and can use managed endpoints or your own containers. It also makes your team responsible for capacity, patches, model upgrades, evaluation and failover.
- ASSESSMENT
The cloud provider becomes a data processor in either pattern: you need a DPA, a documented region, and an answer on cross-region routing and where support staff can access the environment from.
- RECOMMENDATION
Start with the managed-API pattern when the workflow is custom but model operations are not the source of competitive advantage; move to private model serving only if evaluation, volume, portability or the data boundary justifies the extra operations. Your stated technical capability is "unknown".
Self-hosted
STRONG ALTERNATIVEOpen-weight models on infrastructure you operate
- ASSESSMENT
Documents, queries and embeddings stay on machines you own, using an open-weight model whose licence you review. For a brief that involves confidential documents and personal data, that removes a model-API vendor from the data path rather than governing that transfer by contract.
- ASSESSMENT
It costs you the operational work instead: a GPU server, Docker, Linux, backups and a patching routine. Your stated technical capability is "unknown", which is the attribute this option most depends on.
- ASSESSMENT
No processor agreement, subprocessor list or cross-border transfer assessment is needed for the model itself, because no third party processes the content.
- RECOMMENDATION
Recommended where local processing is preferred (you did not say so) and the content is sensitive (confidential documents and personal data).
Enterprise SaaS
CONSIDER IFFinished closed-source cloud product with enterprise controls
- ASSESSMENT
This is a complete vendor application, not a model API: examples include an enterprise assistant, coding copilot or document product with the workflow, interface, connectors and administration already built. It can use closed-source cloud models while requiring no model hosting or application engineering from your team.
- RECOMMENDATION
Choose it when the product already performs the actual workflow and its controls meet your requirements. Do not choose it only because its underlying model is strong: a finished SaaS product is less flexible than building against a managed API when your process, integrations or review steps are organisation-specific.
- ASSESSMENT
Vendor commitments are treated as unverified until we have fetched the page that makes them. Until then this option carries questions to ask, not assurances: a signed data processing agreement covering the data you will actually put in; a documented data residency commitment naming the region, in the contract rather than a blog post; a written no-training commitment for your content, including uploads and connected sources; stated retention periods and a deletion path you can exercise; an administrative audit log you can export, and SSO with group-based access control.
- RECOMMENDATION
No jurisdiction was named, so this is the check rather than the conclusion: compare the vendor's stated processing locations and subprocessor list against the cross-border transfer rules wherever you operate before uploading anything.
03Deployment stacks
- Self-hostedLocal LLM inference server (Ollama / vLLM)vLLM for throughput or Ollama for simplicity, one GPU, and LiteLLM in front as the gateway that issues per-team keys and records spend. The endpoint other recipes point at.
- Self-hostedPrivate ChatGPT on your own serverOllama serving an open-weight model on one GPU, Open WebUI in front of it for accounts, chats and admin controls, both in Docker on a single machine. Deliberately the smallest thing that works: no vector database, no connectors, no cluster.
- Self-hostedPrivate company knowledge base (self-hosted RAG)Open WebUI as the employee interface, vLLM serving a Qwen2.5-14B-Instruct model on a single 24 GB GPU, PostgreSQL with pgvector for chats and embeddings, and OIDC single sign-on — all in Docker on one server in your office or colocation rack. Ollama replaces vLLM for teams under about 20 users; a 48 GB GPU lets you run a 32B model for better answers.
04Tools by hosting option
Self-hosted10
- Self-hostedLibreChatSelf-hosted multi-model chat application with authentication, per-conversation model switching, plugins, file upload and an admin configuration file. Familiar interface for staff moving off consumer tools.
- PlatformLiteLLMGateway that presents one OpenAI-compatible API in front of many providers and local servers, with per-team keys, budgets, rate limits, fallbacks and request logging.
- Libraryllama.cppC++ inference engine for quantised models on CPU, Apple Silicon and GPUs, with a bundled HTTP server. The engine underneath many desktop runtimes and the GGUF quantisation format.
- HybridLM StudioDesktop application for downloading and running open-weight models locally, with a chat interface and a local OpenAI-compatible server. Windows, macOS and Linux.
- Model serverLocalAIDrop-in OpenAI-compatible API server that runs text, embedding, image and audio models locally across several back ends, including CPU-only deployments.
- Model serverOllamaLocal model runtime with a one-command install, a model library and an OpenAI-compatible API. The usual starting point for running open-weight models on a workstation or small server.
- Self-hostedOpen WebUISelf-hosted chat interface for local and hosted models, with user accounts, groups, document upload and built-in retrieval. Runs in Docker against Ollama, vLLM or any OpenAI-compatible endpoint.
- HybridQdrantVector search engine written in Rust with payload filtering, hybrid search, quantisation and snapshots. Runs as a single container or a cluster, or as a managed cloud service.
- Model serverSGLangServing framework for large models with prefix caching and structured-output support, aimed at high-throughput deployments and multi-GPU serving.
- Model servervLLMHigh-throughput GPU inference server using paged attention and continuous batching, with an OpenAI-compatible API. Built for many concurrent users rather than single-session use.
Private cloud12
- PlatformAmazon BedrockAWS service offering models from several providers behind one API in a chosen region, with VPC endpoints, IAM control, guardrails and knowledge bases for retrieval.
- PlatformAzure OpenAI ServiceOpenAI models served from a customer-selected Azure region under Azure commercial terms, with private networking, content filtering and Entra ID integration.
- PlatformGoogle Vertex AIGoogle Cloud platform for Gemini and third-party models with regional endpoints, VPC Service Controls, grounding against your own data and enterprise IAM.
- Self-hostedLibreChatSelf-hosted multi-model chat application with authentication, per-conversation model switching, plugins, file upload and an admin configuration file. Familiar interface for staff moving off consumer tools.
- PlatformLiteLLMGateway that presents one OpenAI-compatible API in front of many providers and local servers, with per-team keys, budgets, rate limits, fallbacks and request logging.
- Model serverLocalAIDrop-in OpenAI-compatible API server that runs text, embedding, image and audio models locally across several back ends, including CPU-only deployments.
- SaaSMistral Le ChatAssistant and API platform from a European vendor, offered as a hosted service and, for enterprise customers, as a deployment inside the customer’s own infrastructure.
- Model serverOllamaLocal model runtime with a one-command install, a model library and an OpenAI-compatible API. The usual starting point for running open-weight models on a workstation or small server.
- Self-hostedOpen WebUISelf-hosted chat interface for local and hosted models, with user accounts, groups, document upload and built-in retrieval. Runs in Docker against Ollama, vLLM or any OpenAI-compatible endpoint.
- HybridQdrantVector search engine written in Rust with payload filtering, hybrid search, quantisation and snapshots. Runs as a single container or a cluster, or as a managed cloud service.
- Model serverSGLangServing framework for large models with prefix caching and structured-output support, aimed at high-throughput deployments and multi-GPU serving.
- Model servervLLMHigh-throughput GPU inference server using paged attention and continuous batching, with an OpenAI-compatible API. Built for many concurrent users rather than single-session use.
Vendor cloud6
- PlatformAmazon BedrockAWS service offering models from several providers behind one API in a chosen region, with VPC endpoints, IAM control, guardrails and knowledge bases for retrieval.
- PlatformAzure OpenAI ServiceOpenAI models served from a customer-selected Azure region under Azure commercial terms, with private networking, content filtering and Entra ID integration.
- PlatformGoogle Vertex AIGoogle Cloud platform for Gemini and third-party models with regional endpoints, VPC Service Controls, grounding against your own data and enterprise IAM.
- PlatformLiteLLMGateway that presents one OpenAI-compatible API in front of many providers and local servers, with per-team keys, budgets, rate limits, fallbacks and request logging.
- SaaSMistral Le ChatAssistant and API platform from a European vendor, offered as a hosted service and, for enterprise customers, as a deployment inside the customer’s own infrastructure.
- HybridQdrantVector search engine written in Rust with payload filtering, hybrid search, quantisation and snapshots. Runs as a single container or a cluster, or as a managed cloud service.
On-premise (enterprise plan)5
- Self-hostedLibreChatSelf-hosted multi-model chat application with authentication, per-conversation model switching, plugins, file upload and an admin configuration file. Familiar interface for staff moving off consumer tools.
- SaaSMistral Le ChatAssistant and API platform from a European vendor, offered as a hosted service and, for enterprise customers, as a deployment inside the customer’s own infrastructure.
- Self-hostedOpen WebUISelf-hosted chat interface for local and hosted models, with user accounts, groups, document upload and built-in retrieval. Runs in Docker against Ollama, vLLM or any OpenAI-compatible endpoint.
- Model serverSGLangServing framework for large models with prefix caching and structured-output support, aimed at high-throughput deployments and multi-GPU serving.
- Model servervLLMHigh-throughput GPU inference server using paged attention and continuous batching, with an OpenAI-compatible API. Built for many concurrent users rather than single-session use.
05Compliance hot spots
This use case usually raises data residency, cross-border transfers, vendor jurisdiction, model training, subprocessors, data processing agreement, confidentiality, logging, personal data, prompt leakage, retention, security, transparency.
- Data residency
- No published jurisdiction page names this topic yet
- Cross-border transfers
- European UnionUnited KingdomChina (mainland)Hong KongTaiwan
- Vendor jurisdiction
- No published jurisdiction page names this topic yet
- Model training
- No published jurisdiction page names this topic yet
- Subprocessors
- No published jurisdiction page names this topic yet
- Data processing agreement
- No published jurisdiction page names this topic yet
- Logging
- European Union
- Retention
- Hong Kong
- Security
- China (mainland)
- Transparency
- United KingdomJapanSouth Korea
06Example questions
Each of these opens the question box with the text already in it. The answer is researched for your organisation, not for this page.
07Related use cases
- InfrastructurePrivate company ChatGPTA chat assistant for staff that behaves like a consumer chatbot but runs under company control: your accounts, your logging, your retention, your choice of model, and optionally your own documents attached.
- InfrastructureLocal LLMRunning an open-weight model on hardware you own — a workstation, a GPU server, a laptop. The question is usually which model fits the memory you have and how many people can use it at once.
Improve this page
Sign in to contribute
From the field
0 deployments · 0 questions
Nobody has reported deploying this here yet, and no question has been opened against this page. Both appear once a reviewer accepts them.