Private LLM

You write and host the application — the interface, the retrieval layer, the database, the access control, the audit log — and call somebody else’s endpoint for the model itself. Two kinds of endpoint sit in this class: a closed model from the vendor that made it (OpenAI, Anthropic, Google Gemini, xAI, DeepSeek, Moonshot, Mistral, Cohere), and an open-weight model served by a managed inference provider (Together, Fireworks, Groq and others).

Those two are not interchangeable. A closed model can only be obtained from its vendor or from a cloud that licenses it, so the model and the supplier are one decision. An open-weight model can be moved: the same weights run on a managed endpoint today and on your own GPU later, which makes the provider a swappable component rather than a dependency — and makes the licence, not the contract, the document that governs what you may do with it.

It is the shortest path to a frontier model: no GPUs to buy, no capacity to plan, no serving stack to patch, and you can change model with a configuration line. What it costs instead is a per-token bill that scales with use and a dependency on one supplier’s availability, pricing and deprecation schedule.

What the supplier sees is the whole prompt: the user's question, the system instructions, and every passage your retrieval layer attached to it — which is usually the most sensitive part, because retrieval is what puts internal documents into the request. For a brief that involves confidential documents and personal data, that is the attribute to check first. What stays yours is everything else: the application, the retrieval index, the identity system, the logs, and the record of who asked what.

Nothing about a supplier's handling of your data is assumed here. Before real content goes near an endpoint, verify against that supplier's own documents: the data processing agreement, and that it covers the API rather than only the consumer product; whether zero-data-retention or an equivalent no-logging mode is available on your account, and on which endpoints; the default retention period for prompts and outputs, and what triggers human review; the region the request is served from, and whether that is a commitment or a routing preference; the current subprocessor list and how you are notified when it changes; the supported-countries page, which decides whether you may use the service at all.

Documents, queries and embeddings stay on machines you own, using an open-weight model whose licence you review. For a brief that involves confidential documents and personal data, that removes a model-API vendor from the data path rather than governing that transfer by contract.

It costs you the operational work instead: a GPU server, Docker, Linux, backups and a patching routine. Your stated technical capability is "unknown", which is the attribute this option most depends on.

No processor agreement, subprocessor list or cross-border transfer assessment is needed for the model itself, because no third party processes the content.