NVIDIA TensorRT-LLM, Amazon Bedrock vs Baseten

The registry records NVIDIA TensorRT-LLM as available on self-hosted or private cloud or on-premise enterprise; it is scored here as self-hosted.

The registry records Amazon Bedrock as available on vendor cloud or private cloud; it is scored here as enterprise SaaS.

The registry records Baseten as available on vendor_api or private cloud; it is scored here as a model vendor's API.

For this brief — confidential documents — self-hosted is a strong alternative. Change the brief and this verdict changes.

For this brief — confidential documents — enterprise SaaS is conditional: worth it only if the conditions in the options section hold. Change the brief and this verdict changes.

For this brief — confidential documents — a model vendor's API is a strong alternative. Change the brief and this verdict changes.

Covers your main objective (private LLM) and 1 of 1 secondary objectives (local LLM).

Covers your main objective (private LLM) and 0 of 1 secondary objectives.

Covers your main objective (private LLM) and 1 of 1 secondary objectives (local LLM).

You stated no hosting preference, so no option is favoured on that basis. No hard restriction on where processing happens was stated. You did not name a cloud you already run on, so no design is favoured on that basis.

You stated no hosting preference, so no option is favoured on that basis. No hard restriction on where processing happens was stated. You did not name a cloud you already run on, so no design is favoured on that basis.

You stated no hosting preference, so no option is favoured on that basis. No hard restriction on where processing happens was stated. You did not name a cloud you already run on, so no design is favoured on that basis.

External data transfer: none. No catalogue deployment names NVIDIA TensorRT-LLM, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief. No edge in this design crosses out of the company network.

External data transfer: some. Computed from “Assistant on Amazon Bedrock”, the catalogue deployment that names Amazon Bedrock and best matches this brief. Confidential content leaves your premises for your own cloud tenancy ("VPC / PrivateLink"). You keep control of the account; the provider is a processor, so a DPA and a documented region apply.

External data transfer: yes. No catalogue deployment names Baseten, so this is read from “Agent workflow on a model vendor’s API”, the best-matching a model vendor's API deployment for this brief. Confidential content leaves your control on the Tools + retrieval (your APIs and data) → Vendor model API link.

Self-hosted: the content stays wherever you run the server, so residency is a property of your own infrastructure rather than a vendor commitment.

We hold no fetched residency commitment for Amazon Bedrock. That is a question to put to the vendor, not an assumption to make about them. The vendor is headquartered in United States, which is where a transfer question starts, not where it ends.

We hold no fetched residency commitment for Baseten. That is a question to put to the vendor, not an assumption to make about them. The vendor is headquartered in United States, which is where a transfer question starts, not where it ends.

Run on your own infrastructure, so the vendor questions apply only to whatever you still buy. We hold none of the four commitments for this subject, and for a self-hosted deployment none of them is required.

We have fetched none of the four commitments for Amazon Bedrock: data processing agreement, subprocessor list, training on customer data, zero retention option. Each is a question for the vendor, and until it is answered it is unknown rather than absent.

We have fetched none of the four commitments for Baseten: data processing agreement, subprocessor list, training on customer data, zero retention option. Each is a question for the vendor, and until it is answered it is unknown rather than absent.

2–6 FDE-days to implement. No catalogue deployment names NVIDIA TensorRT-LLM, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief.

6–16 FDE-days to implement. Computed from “Assistant on Amazon Bedrock”, the catalogue deployment that names Amazon Bedrock and best matches this brief.

8–22 FDE-days to implement. No catalogue deployment names Baseten, so this is read from “Agent workflow on a model vendor’s API”, the best-matching a model vendor's API deployment for this brief.

You own GPU drivers and the model server, operating-system patching and backups. Assessed difficulty 2/5 is the shape of that work; someone has to hold it after go-live.

The vendor runs the service. Your standing work is access control, the policy staff work under, and re-reading the contract and subprocessor list when they change — not patching or capacity.

You own the identity integration and joiner/leaver process, the connectors and their credentials, operating-system patching and backups. Assessed difficulty 3/5 is the shape of that work; someone has to hold it after go-live.

2/5. Starting point 2/5: software you install, run and keep running on your own machines. +0.5 You run the model server yourself: GPU drivers, quantisation choice, memory headroom and restarts are all yours to own.

4/5. Starting point 2/5: your own tenancy to build in, but no hardware to buy or rack. +0.5 You run the model server yourself: GPU drivers, quantisation choice, memory headroom and restarts are all yours to own.

3/5. Starting point 2/5: software you install, run and keep running on your own machines. -0.5 The model runs on the vendor’s infrastructure: there is no capacity planning, no GPU driver, no model upgrade window and no failover for you to design.

USD 5,500 – 21,000 — one-off — for NVIDIA TensorRT-LLM at this headcount. No catalogue deployment names NVIDIA TensorRT-LLM, so this is read from “Local LLM inference server (Ollama / vLLM)”, the best-matching self-hosted deployment for this brief.

USD 4,600 – 31,000 — one-off — for Amazon Bedrock at this headcount. Computed from “Assistant on Amazon Bedrock”, the catalogue deployment that names Amazon Bedrock and best matches this brief. Licences are not in that figure: no per-seat price has been read from the vendor’s own pricing page, so the software line is excluded from the total rather than guessed. Read it as implementation only.

USD 6,100 – 43,000 — one-off — for Baseten at this headcount. No catalogue deployment names Baseten, so this is read from “Agent workflow on a model vendor’s API”, the best-matching a model vendor's API deployment for this brief.

The registry files NVIDIA TensorRT-LLM as open source, so the interface, retrieval behaviour and the model behind it can be changed — subject to the licence, which is a fetched fact on its own page.

Amazon Bedrock is a vendor product: you configure what the vendor exposes — policies, connectors, retention settings — and nothing below that line.

Baseten is a vendor product: you configure what the vendor exposes — policies, connectors, retention settings — and nothing below that line.

7 issues spotted for this brief, 0 of them legal requirements; 0 carry a quote from the instrument they rest on. Leading with: Self-hosting moves the security obligation to you.

8 issues spotted for this brief, 0 of them legal requirements; 0 carry a quote from the instrument they rest on. Leading with: Confidentiality duties bind independently of data protection law.

9 issues spotted for this brief, 0 of them legal requirements; 0 carry a quote from the instrument they rest on. Leading with: Confidentiality duties bind independently of data protection law.