Compare
LMDeploy, Hugging Face Inference Endpoints, llama.cpp vs Jan
Every row below is computed for the brief in section 01. Nothing here is a universal score.
- Source
- Registry and rules engines over one brief — nothing on this page is anchored in a fetched document yet
- Evidence
- none on this page — it links to the pages that hold it
01The brief this answers
No context was supplied, so this is a generic reading. Every figure below would move for a real brief.
The same tool can score differently for a different brief.
- Objective
- Local LLM
- Also
- Private LLM · Document Q&A
- Jurisdictions
- none stated
- Industry
- unknown
- Employees
- unknown
- Data
- confidential yes · privileged unknown · personal likely · sensitive unknown
- Local processing
- not required
- Cloud allowed
- unknown
- Users
- unknown
- Capability
- unknown
- Preference
- no preference
- Data risk
- high
interpreted by: rules (deterministic mode) · confidence 40%
unresolved: organisation.industry, requirements.dataResidency, requirements.existingStack, budget, jurisdictions.primary, organisation.employees, data sensitivity, deploymentPreference, technicalCapability
Not accounted for
Nothing in this comparison depends on your industry, any residency requirement, the systems you already run, your budget, the jurisdiction you operate in, your headcount, how sensitive the material is, a hosting preference or the technical capability you have in-house — none of it was supplied. Where a figure would move for one of those, it is stated as an assumption rather than hidden inside the number.
02Comparison
| Dimension | LMDeployToolSTRONG ALTERNATIVEfor self-hosted under this brief | Hugging Face Inference EndpointsToolSTRONG ALTERNATIVEfor a model vendor's API under this brief | llama.cppToolSTRONG ALTERNATIVEfor self-hosted under this brief | JanToolSTRONG ALTERNATIVEfor self-hosted under this brief |
|---|---|---|---|---|
| Deployment modelRegistry entry: how this is delivered, and which solution class it is scored as. | Self-hosted The registry records LMDeploy as available on self-hosted or private cloud; it is scored here as self-hosted. ASSESSMENT | Model API The registry records Hugging Face Inference Endpoints as available on vendor_api or private cloud; it is scored here as a model vendor's API. ASSESSMENT | Self-hosted The registry records llama.cpp as available only on self-hosted; it is scored here as self-hosted. ASSESSMENT | Self-hosted The registry records Jan as available only on self-hosted; it is scored here as self-hosted. ASSESSMENT |
| Verdict for this briefSolution-class analysis (buildOptions) over this brief. The verdict is on the class of answer, not on the product. | STRONG ALTERNATIVE For this brief — confidential documents — self-hosted is a strong alternative. Change the brief and this verdict changes. RECOMMENDATION | STRONG ALTERNATIVE For this brief — confidential documents — a model vendor's API is a strong alternative. Change the brief and this verdict changes. RECOMMENDATION | STRONG ALTERNATIVE For this brief — confidential documents — self-hosted is a strong alternative. Change the brief and this verdict changes. RECOMMENDATION | STRONG ALTERNATIVE For this brief — confidential documents — self-hosted is a strong alternative. Change the brief and this verdict changes. RECOMMENDATION |
| Functional fitOntology overlap between the objective in this brief and the use cases the registry records for the subject (matchRecipes). | 80% Covers your main objective (local LLM) and 1 of 2 secondary objectives (private LLM). ASSESSMENT | 100% Covers your main objective (local LLM) and 2 of 2 secondary objectives (private LLM, document Q&A). ASSESSMENT | 80% Covers your main objective (local LLM) and 1 of 2 secondary objectives (private LLM). ASSESSMENT | 100% Covers your main objective (local LLM) and 2 of 2 secondary objectives (private LLM, document Q&A). ASSESSMENT |
| Deployment fitHosting preference, data egress, headcount band and technical capability, weighted 0.30 / 0.35 / 0.15 / 0.20 (matchRecipes). | 74% You stated no hosting preference, so no option is favoured on that basis. No hard restriction on where processing happens was stated. You did not name a cloud you already run on, so no design is favoured on that basis. ASSESSMENT | 74% You stated no hosting preference, so no option is favoured on that basis. No hard restriction on where processing happens was stated. You did not name a cloud you already run on, so no design is favoured on that basis. ASSESSMENT | 74% You stated no hosting preference, so no option is favoured on that basis. No hard restriction on where processing happens was stated. You did not name a cloud you already run on, so no design is favoured on that basis. ASSESSMENT | 74% You stated no hosting preference, so no option is favoured on that basis. No hard restriction on where processing happens was stated. You did not name a cloud you already run on, so no design is favoured on that basis. ASSESSMENT |
| Data controlExternal transfer computed from the architecture graph of the deployment (computeExternalTransfer) — what actually crosses out of your network. | none External data transfer: none. No catalogue deployment names LMDeploy, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. No edge in this design crosses out of the company network. ASSESSMENT | yes External data transfer: yes. No catalogue deployment names Hugging Face Inference Endpoints, so this is read from “Assistant on a model vendor’s API”, the best-matching a model vendor's API deployment for this brief. Confidential content leaves your control on the Retrieval layer → Vendor model API link. ASSESSMENT | none External data transfer: none. No catalogue deployment names llama.cpp, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. No edge in this design crosses out of the company network. ASSESSMENT | none External data transfer: none. No catalogue deployment names Jan, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. No edge in this design crosses out of the company network. ASSESSMENT |
| Data residencyVendor documents we have fetched. A residency commitment we have not read is “unknown” — never assumed (CONTENT-RULES §5). | you decide Self-hosted: the content stays wherever you run the server, so residency is a property of your own infrastructure rather than a vendor commitment. ASSESSMENT | unknown We hold no fetched residency commitment for Hugging Face Inference Endpoints. That is a question to put to the vendor, not an assumption to make about them. The vendor is headquartered in United States, which is where a transfer question starts, not where it ends. ASSESSMENT | you decide Self-hosted: the content stays wherever you run the server, so residency is a property of your own infrastructure rather than a vendor commitment. ASSESSMENT | you decide Self-hosted: the content stays wherever you run the server, so residency is a property of your own infrastructure rather than a vendor commitment. ASSESSMENT |
| Governance controlsThe four commitments a processor is asked for — DPA, subprocessor list, no training on your content, zero retention — counted against fetched vendor documents. | not applicable Run on your own infrastructure, so the vendor questions apply only to whatever you still buy. We hold none of the four commitments for this subject, and for a self-hosted deployment none of them is required. ASSESSMENT | unknown We have fetched none of the four commitments for Hugging Face Inference Endpoints: data processing agreement, subprocessor list, training on customer data, zero retention option. Each is a question for the vendor, and until it is answered it is unknown rather than absent. ASSESSMENT | not applicable Run on your own infrastructure, so the vendor questions apply only to whatever you still buy. We hold none of the four commitments for this subject, and for a self-hosted deployment none of them is required. ASSESSMENT | not applicable Run on your own infrastructure, so the vendor questions apply only to whatever you still buy. We hold none of the four commitments for this subject, and for a self-hosted deployment none of them is required. ASSESSMENT |
| Integration effortThe FDE-day band the catalogue deployment was costed from (RECIPE_IMPLEMENTATION_DAYS). | 2–6 FDE-days 2–6 FDE-days to implement. No catalogue deployment names LMDeploy, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. ASSESSMENT | 5–14 FDE-days 5–14 FDE-days to implement. No catalogue deployment names Hugging Face Inference Endpoints, so this is read from “Assistant on a model vendor’s API”, the best-matching a model vendor's API deployment for this brief. ASSESSMENT | 2–6 FDE-days 2–6 FDE-days to implement. No catalogue deployment names llama.cpp, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. ASSESSMENT | 2–6 FDE-days 2–6 FDE-days to implement. No catalogue deployment names Jan, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. ASSESSMENT |
| Maintenance burdenWho operates the result, read from the solution class and the difficulty factors the deployment carries. | you operate it You own GPU drivers and the model server, operating-system patching and backups. Assessed difficulty 2/5 is the shape of that work; someone has to hold it after go-live. ASSESSMENT | you operate it You own the identity integration and joiner/leaver process, the connectors and their credentials, operating-system patching and backups. Assessed difficulty 2/5 is the shape of that work; someone has to hold it after go-live. ASSESSMENT | you operate it You own GPU drivers and the model server, operating-system patching and backups. Assessed difficulty 2/5 is the shape of that work; someone has to hold it after go-live. ASSESSMENT | you operate it You own GPU drivers and the model server, operating-system patching and backups. Assessed difficulty 2/5 is the shape of that work; someone has to hold it after go-live. ASSESSMENT |
| Deployment difficultyDifficulty 1–5 for the deployment and for this team (scoreDifficulty): the same stack scores lower for an organisation with its own engineers. | 2/5 2/5. Starting point 2/5: software you install, run and keep running on your own machines. +0.5 You run the model server yourself: GPU drivers, quantisation choice, memory headroom and restarts are all yours to own. ASSESSMENT | 2/5 2/5. Starting point 2/5: software you install, run and keep running on your own machines. -0.5 The model runs on the vendor’s infrastructure: there is no capacity planning, no GPU driver, no model upgrade window and no failover for you to design. ASSESSMENT | 2/5 2/5. Starting point 2/5: software you install, run and keep running on your own machines. +0.5 You run the model server yourself: GPU drivers, quantisation choice, memory headroom and restarts are all yours to own. ASSESSMENT | 2/5 2/5. Starting point 2/5: software you install, run and keep running on your own machines. +0.5 You run the model server yourself: GPU drivers, quantisation choice, memory headroom and restarts are all yours to own. ASSESSMENT |
| Estimated costIndicative cost for this headcount (estimateCost), plus any per-token price we have actually fetched. A price we have not read is not printed. | USD 5,500 – 21,000 USD 5,500 – 21,000 — one-off — for LMDeploy at this headcount. No catalogue deployment names LMDeploy, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. ASSESSMENT | USD 3,800 – 27,000 USD 3,800 – 27,000 — one-off — for Hugging Face Inference Endpoints at this headcount. No catalogue deployment names Hugging Face Inference Endpoints, so this is read from “Assistant on a model vendor’s API”, the best-matching a model vendor's API deployment for this brief. ASSESSMENT | USD 5,500 – 21,000 USD 5,500 – 21,000 — one-off — for llama.cpp at this headcount. No catalogue deployment names llama.cpp, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. ASSESSMENT | USD 5,500 – 21,000 USD 5,500 – 21,000 — one-off — for Jan at this headcount. No catalogue deployment names Jan, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. ASSESSMENT |
| CustomisationRegistry entry: whether the source is open and where the deployment can be changed. | source available The registry files LMDeploy as open source, so the interface, retrieval behaviour and the model behind it can be changed — subject to the licence, which is a fetched fact on its own page. ASSESSMENT | vendor-configured Hugging Face Inference Endpoints is a vendor product: you configure what the vendor exposes — policies, connectors, retention settings — and nothing below that line. ASSESSMENT | source available The registry files llama.cpp as open source, so the interface, retrieval behaviour and the model behind it can be changed — subject to the licence, which is a fetched fact on its own page. ASSESSMENT | source available The registry files Jan as open source, so the interface, retrieval behaviour and the model behind it can be changed — subject to the licence, which is a fetched fact on its own page. ASSESSMENT |
| Relevant compliance evidenceThe compliance engine over this brief and this subject (assessCompliance), with each issue cited to the instrument it quotes where the page was fetched. | 7 issues · 0 cited 7 issues spotted for this brief, 0 of them legal requirements; 0 carry a quote from the instrument they rest on. Leading with: Self-hosting moves the security obligation to you. ASSESSMENT | 9 issues · 0 cited 9 issues spotted for this brief, 0 of them legal requirements; 0 carry a quote from the instrument they rest on. Leading with: Confidentiality duties bind independently of data protection law. ASSESSMENT | 7 issues · 0 cited 7 issues spotted for this brief, 0 of them legal requirements; 0 carry a quote from the instrument they rest on. Leading with: Self-hosting moves the security obligation to you. ASSESSMENT | 7 issues · 0 cited 7 issues spotted for this brief, 0 of them legal requirements; 0 carry a quote from the instrument they rest on. Leading with: Self-hosting moves the security obligation to you. ASSESSMENT |
Comparing
- LMDeployremove LMDeploy from this comparison
- Hugging Face Inference Endpointsremove Hugging Face Inference Endpoints from this comparison
- llama.cppremove llama.cpp from this comparison
- Janremove Jan from this comparison
Add one that serves the same objective
This table is full at 4 subjects. Remove one to add another — beyond four, the columns stop being readable and the comparison stops being one.
- Amazon SageMaker AI
- AnythingLLM
- Baseten
- Continue
- DeepSeek API
- Fireworks AI
- LiteLLM
- LM Studio
03What this does not tell you
- WARNINGCompare no jurisdictioncompare_no_jurisdiction
No jurisdiction was supplied, so only the cross-cutting rules ran. A comparison for a regulated deployment should name one.
- MINORCompare objective derivedcompare_objective_derived
No objective was given, so functional fit is measured against local LLM — the use case most of these subjects share. Add one to the link to measure a different job.
- MINORCompare recipe substitutedcompare_recipe_substituted
No catalogue deployment names LMDeploy, so its cost, effort and difficulty are read from “Local LLM inference server (Ollama / vLLM)”, the closest self-hosted deployment for this brief.
- MINORCompare recipe substitutedcompare_recipe_substituted
No catalogue deployment names Hugging Face Inference Endpoints, so its cost, effort and difficulty are read from “Assistant on a model vendor’s API”, the closest a model vendor's API deployment for this brief.
- MINORCompare recipe substitutedcompare_recipe_substituted
No catalogue deployment names llama.cpp, so its cost, effort and difficulty are read from “Local LLM inference server (Ollama / vLLM)”, the closest self-hosted deployment for this brief.
- MINORCompare recipe substitutedcompare_recipe_substituted
No catalogue deployment names Jan, so its cost, effort and difficulty are read from “Local LLM inference server (Ollama / vLLM)”, the closest self-hosted deployment for this brief.
Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.
04Evidence
No sources were recorded for this answer. Nothing on this page should be treated as verified.