Compliance
NVIDIA TensorRT-LLM in United States
Structured issue-spotting for deploying NVIDIA TensorRT-LLM in United States, read against the rules for that jurisdiction.
- Source
- Rules engine over a generic brief — no anchored quote on this page
- Verified
- Evidence not verified
- Confidence
- Low
Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.
01What this reading assumes
- Jurisdiction
- United States
- Principal framework
- Sectoral, not omnibus. FTC Act §5 (15 U.S.C. 45) is the general backstop for unfair or deceptive AI and data practices. HIPAA (45 CFR Part 164) covers protected health information; the GLBA Safeguards Rule (16 CFR Part 314) covers financial institutions; FCRA (15 U.S.C. 1681) governs consumer reports and adverse-action notices, which is the statute AI credit, tenant and employment screening most often engages; the COPPA Rule (16 CFR Part 312) covers under-13 data, and the 2025 amendments’ compliance deadline has passed. There is no federal cross-border transfer regime for ordinary personal data. State law supplies what the federal layer does not: California’s CCPA and the CPPA’s ADMT regulations, Colorado’s automated decision-making statute from 2027, Illinois BIPA and the Illinois Human Rights Act AI amendment, Texas TRAIGA, Connecticut Public Act 26-15 and New York City Local Law 144.
- Delivery assessed
- Self-hosted
- Data leaves the network
- no
- Vendor home jurisdiction
- United States
- Verified vendor positions
- none — every vendor position below is a question, not an assurance
- Rules evaluated
- 41
- Rules fired
- 11
Assumptions about use
- An internal deployment used by employees, not a public-facing product.
- A person reads the output before acting on it — but that is not recorded, so the engine reports it as a gap rather than assuming it.
- No significant automated decision is taken about a person by the system alone.
02Issues to work through
Cross Cutting
Self-hosting moves the security obligation to you
Keeping data on your own hardware answers the transfer question and creates an operations question. Patching, backups, key management, monitoring and incident response are now yours, and an unpatched inference server on the office network is a worse outcome than a well-run vendor.
- Required checks
- Name the person responsible for patching each component, and the cadence.
- Confirm backups exist, are encrypted, and have been restored at least once.
- Confirm there is an incident response path that includes this system.
- Technical controls
- Encrypt at rest and in transit, including between the application and the inference server.
- Centralise authentication through the existing identity provider rather than local accounts.
- Keep an audit log of who queried what, and protect it from the people it records.
- Subscribe to security advisories for each component and track upgrade lag.
Being able to reconstruct a decision months later
The question that arrives after a complaint is what the system was shown and what it produced on a particular day. Models change, prompts change, and indexes are rebuilt, so the answer has to be recorded at the time. Without it, the only available response is that the output cannot be reproduced.
- Required checks
- Decide what is recorded per interaction: model and version, prompt template version, retrieved document ids, output, reviewer and outcome.
- Set how long those records are kept, balanced against the retention duties that also apply to them.
- Vendor questions
- Does the vendor pin model versions, and how much notice is given before a model is retired or changed?
- Technical controls
- Version prompt templates in source control and log the version used.
- Log the model identifier and version returned by the provider, not the one you requested.
Confidentiality duties bind independently of data protection law
Material can be entirely free of personal data and still be the material a contract stops you disclosing. Client retainers, non-disclosure agreements, supplier contracts and common-law duties are the usual sources, and several of them require consent before a third party processes the material at all — which a model API call is.
- Required checks
- Review the confidentiality clauses in the contracts covering the material going into the system.
- Identify any contract requiring notice or consent before a subcontractor processes the material.
- Decide whether the deployment needs a confidentiality carve-out negotiated into new contracts.
- Vendor questions
- Will the vendor accept a confidentiality undertaking beyond its standard terms?
- Which staff at the vendor can access customer content, under what controls?
- Technical controls
- Segregate the most sensitive corpora into an index that the general assistant cannot reach.
Human review required — take this to your counsel
We were not told whether a person reviews the output
Where output influences a decision about a person, the reviewer has to be able to disagree with it. That needs three things a rubber-stamp review lacks: enough information to judge, enough time to judge, and an override that is used often enough to be real. Design it before the volume makes it impossible.
- Required checks
- Name the role that reviews the output and what they see when they do.
- Decide what evidence is retained about each review, so the practice can be shown to exist.
- Set a threshold below which the system must not act without review.
- Vendor questions
- Does the product expose the retrieved context and the confidence behind a suggestion, or only the answer?
- Technical controls
- Show the reviewer the retrieved sources next to the suggestion, not the suggestion alone.
- Record the reviewer’s decision, including overrides, as part of the audit trail.
Human review required — take this to your counsel
An AI deployment creates new copies of the data
Vector indexes, prompt logs, completion caches, evaluation datasets, fine-tuning checkpoints and backups are all copies of the source material in places the existing retention schedule does not mention. Deletion requests are the moment this is discovered, because deleting the source document does not delete its embedding.
- Required checks
- List every store the deployment creates and add each to the retention schedule.
- Establish how a deletion request propagates to the index, the caches and the logs.
- Establish how long backups keep material that has been deleted from the live system.
- Vendor questions
- What does the vendor retain, where, and for how long after we delete our copy?
- Technical controls
- Store the source document id with every embedding so deletion can cascade.
- Set time-to-live on prompt and completion logs rather than relying on manual cleanup.
Who can reach the model, the index and the weights
A self-hosted stack has three access surfaces that are easy to leave open: the inference endpoint, the vector index, and the weights on disk. Retrieval also carries an authorisation problem an ordinary application does not have — the index must not return a document to someone who could not open it in the source system.
- Required checks
- Confirm the inference endpoint is not reachable from outside the network and requires authentication.
- Confirm retrieval filters by the requesting user’s permissions, not only by relevance.
- Confirm who can read the model files and the index volume at the operating-system level.
- Technical controls
- Bind the inference server to a private interface and put an authenticating proxy in front of it.
- Carry document-level access control into the index and enforce it at query time.
- Encrypt the volume holding the weights and the index, and restrict it to the service account.
- Rotate API keys and keep them out of client-side code and container images.
What ends up in a prompt, and where it goes next
Even with inference inside the network, prompts and retrieved context accumulate in logs, traces and caches, and system prompts can often be extracted from the output. The leak path is internal rather than external, but it is still a copy of the source material in a new place.
- Required checks
- Write down which categories of information may be entered into a prompt, and tell people.
- Establish what the system prompt contains and whether disclosing it would matter.
- Establish which shadow tools staff are already using; the policy has to name the permitted ones.
- Vendor questions
- Are prompts and completions retained, for how long, and can retention be set to zero?
- Are prompts used for abuse monitoring, and if so who can read them and for how long?
- Technical controls
- Redact or block high-risk patterns before the prompt leaves the application.
- Keep prompt and completion logs out of general-purpose observability tools.
- Set an explicit retention period on prompt logs and enforce it.
United States
Tell us which states, because the AI duties are state duties and they disagree
The binding AI obligations on a private deployer in the United States are state obligations — bias audits, automated decision-making notices and opt-outs, biometric consent, generative AI disclosure — and they do not agree with one another. This brief does not name a state, so we cannot say which apply. A federal executive order now directs the Justice Department to challenge state AI laws, but a law being litigated is still a law in force.
- Required checks
- Name the states where the people affected by the system live and where the organisation does business.
- Design to the strictest rule that reaches you rather than to a per-state matrix, unless the cost of the strictest rule is real.
- Track the federal preemption litigation, but do not plan on it succeeding.
There is no federal privacy statute to comply with — work out which state law reaches you
The United States has no general federal privacy statute. What applies to a deployment is a sectoral statute if the sector is regulated, plus whichever state privacy laws reach the organisation by revenue, record count or the residence of the people in the data. That is a scoping exercise, not a compliance checklist, and it has to happen before any control is designed — the answer differs for a California consumer, an Illinois employee and a Texas customer.
- Required checks
- List the states whose residents appear in the data, and check each state law’s applicability thresholds against the organisation’s revenue and record counts.
- Decide whether a sectoral statute applies — health, financial, education, children — because that changes the analysis before any state law does.
- Record the states you concluded do not reach you, with the threshold that got you there, so the decision can be revisited when the numbers change.
- Technical controls
- Capture the state of residence where it is already known, so scoping is a query rather than a guess.
FTC Act §5 reaches what you say about the AI, not just what it does
Section 5 of the Federal Trade Commission Act declares unfair or deceptive acts or practices in or affecting commerce unlawful. For an AI deployment that means the marketing copy, the accuracy claims, the privacy policy and the statement about whether customer data trains a model are all in scope, and a claim the system cannot support is the exposure — regardless of whether any AI-specific rule applies.
- Required checks
- Match every public accuracy or capability claim to a measurement you could produce if asked.
- Check the privacy policy actually describes what happens to prompts, retrieved documents and logs.
- Confirm nothing published says data is not used for training unless the vendor has committed to that in writing.
- Vendor questions
- Will you state in the contract that customer content is not used to train or improve your models?
The NIST AI Risk Management Framework is voluntary until a contract names it
The NIST AI Risk Management Framework is intended for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, use and evaluation of AI systems. It carries no penalty of its own. It becomes binding through the back door: customer contracts, state statutes and federal acquisition terms name it, so a deployment that ignores it often has to retrofit the artefacts later.
- Required checks
- Check whether any customer contract or state rule already names the framework or its Generative AI Profile.
- Produce the Govern, Map, Measure and Manage artefacts as the project runs, rather than reconstructing them for an audit.
- Vendor questions
- Have you mapped your product to the AI Risk Management Framework, and will you share that mapping?
Structured issue-spotting to support your own review — not legal advice. Verify against the cited primary sources and your counsel.
03What this reading does not know
- Whether any of the data falls into a special or sensitive category.
- Whether any material is covered by legal professional privilege.
- Whether a person reviews the output before it is acted on.
04Instruments these issues point at
- statuteFTC Act §5Declares unfair or deceptive acts or practices in commerce unlawful. The general federal backstop for overstated AI capability claims, undisclosed AI use and inadequate data practices — it reaches an AI deployment through what the deployer says about it, not through a technology-specific duty.
- standardNIST AI RMFVoluntary framework structured around Govern, Map, Measure and Manage. The de facto reference for US AI governance programmes, and frequently named in contracts and in state law — which is how a voluntary framework becomes a contractual obligation.
05Vendor documents being watched
- Data processing agreementhttps://www.nvidia.com/en-us/agreements/data-processing-addendum/nvidia-cloud-services-data-processing-addendum/not yet fetched
- Privacy policyhttps://www.nvidia.com/en-us/about-nvidia/privacy-policy/not yet fetched
- Security pagehttps://www.nvidia.com/privacy-center/not yet fetched
- Terms of servicehttps://developer.nvidia.com/legal/termsnot yet fetched
06Ask about your own deployment
This page reads the rules against a generic organisation. Your size, industry, data and existing contracts change which of these issues matter and which fall away.