A private LLM is an evidence-backed data boundary
A private large language model, or LLM, is one whose handling of your data is bounded by verifiable controls. Where the model runs is only one of those controls. A deployment labelled private can still retain prompts, export telemetry, expose logs, use provider-held keys or fail to delete derived data.
The object to assess is the complete system around the model. That system receives prompts, performs inference, meaning it generates an output from a trained model, and creates secondary records such as logs, caches and safety classifications. Privacy depends on what happens to every copy.
A hospital's compliance officer reading a vendor's one-page "private LLM" brochure has no way to tell, from that label alone, whether a support engineer at the vendor can open a transcript of a patient conversation to debug a ticket. The brochure describes where the model sits, not who else can read what passed through it.
This distinction changes the buying decision. "On-premise" describes software running on infrastructure controlled by the buyer. "Dedicated cloud" describes infrastructure reserved for one customer. Neither description establishes who can read the data, why it is retained or whether deletion reaches every store.
The constraint map turns a privacy claim into testable controls
A useful assessment starts with the required boundary, then asks for evidence that each data path stays inside it. The following constraint map covers the controls that a hosting label leaves unanswered.
| Constraint | What must be established | Evidence to request |
|---|---|---|
| Access | Which people, software identities and external service providers can read prompts, outputs and derived records | Identity policies, role assignments, support-access procedures and a list of subprocessors, meaning companies engaged to process data for the provider |
| Retention | How long each copy persists, including exceptions triggered by abuse or safety systems | A retention schedule covering prompts, outputs, logs, caches, backups and classifications |
| Training use | Whether submitted data can update the model or any related model | Product-tier terms, opt-in settings and a contractual prohibition where required |
| Egress | Whether data leaves the approved network, region or legal jurisdiction | Network destinations, regional processing terms and outbound traffic controls |
| Logs | What request content or metadata is recorded for operations, security and debugging | Logging configuration, sample records, access rules and retention settings |
| Keys | Who controls the encryption keys that protect stored data, and who can decrypt it during processing | Key ownership, permission policies, rotation procedures and runtime decryption design |
| Deletion | Which original and derived records a deletion request reaches | A deletion procedure, store inventory, completion record and documented exceptions |
These constraints are independent. A deployment can satisfy the training-use row and fail the retention row. It can satisfy the key-ownership row while allowing the service to decrypt every request during inference.
Provider terms show why each row needs its own answer. On the paid tier of one application programming interface, or API, Google says prompts and responses are not used to improve its products. The free tier of the same API says submitted content and generated responses are used to improve Google products and machine-learning technologies. The training outcome changes with the commercial tier, not with a broad description of where the service is hosted. 1
"No training" also does not mean "no storage." OpenAI states that it does not train on customer data by default, while its API inputs and outputs may be retained for up to 30 days to provide the service and identify abuse. 2 A procurement team that reads the "no training" line as the whole answer and moves on to the next vendor question has skipped the 30-day retention window sitting one paragraph below it.
Exceptions need the same scrutiny as the standard window. Anthropic says content flagged by automated trust and safety systems may be retained for up to two years, while the associated classification score may be retained for up to seven years. 3
A buyer therefore needs a retention answer for ordinary requests, flagged requests, operational records and backups. A single duration in a sales response does not cover that map.
Follow the data rather than the model file
The assessment unit is one request from collection to deletion. Begin where the application obtains the text. Record each component that receives the raw prompt, a transformed prompt, the output or metadata derived from either.
A typical path includes an application, an authentication layer, a model gateway, the inference service and an application database. It may also include monitoring tools, content filters, support systems and backup storage. Every additional recipient creates another access, retention, logging and deletion decision.
Retrieval-augmented generation, or RAG, adds documents retrieved from an external knowledge store to the model prompt. The model endpoint may satisfy its privacy commitments while the retrieval service returns documents that the requesting user was never allowed to see. How to preserve RAG access controls end to end explains why the user's permissions must survive retrieval, prompt construction and output delivery.
For each component, identify the data entering it, the data leaving it, the identities permitted to access it and the records it creates. Then attach evidence. A network diagram supports the egress claim. A configuration export supports the logging claim. Contract terms support the training-use claim. None substitutes for the others.
This process also separates privacy from model quality. A system can keep data inside the required boundary and still invent an answer, expose an authorised document in the wrong context or trigger an unsafe action. The operational controls for those failures belong in a separate review of What happens when the AI gets one wrong?.
Each deployment option closes different parts of the map
A shared hosted API transfers model operations to a provider. Its privacy case rests on contractual terms, service configuration, network controls and the provider's operational evidence. It can be suitable when those commitments match the data policy. The buyer still depends on the provider to perform commitments that cannot be inspected directly from the client side.
A dedicated cloud instance can reduce exposure to other tenants, meaning customers sharing a provider's infrastructure. It does not establish a retention period, prohibit training use or prevent provider personnel from receiving authorised support access. Isolation answers the access question only to the extent documented by the provider's design and permissions.
Self-hosting gives the operator direct control over the inference infrastructure. It also makes the operator responsible for patching, identity management, logs, backups, model files and deletion workflows. Software running locally can still transmit usage information. Hugging Face documents that some of its client libraries collect usage data by default for monitoring, debugging and feature prioritisation, with an explicit setting required to disable that behavior. 4
An isolated deployment with outbound network traffic blocked provides stronger evidence for egress because the boundary is enforced by the network. It also moves model distribution, updates, monitoring and incident response inside that boundary. The compute and operational implications should be evaluated before treating local deployment as the default. How much hardware does your on-premise LLM need? covers that separate capacity decision.
Local operation also does not establish compliance with a sector rule by itself. Compliance depends on the data, purpose, safeguards, agreements and operating procedures around the system. Local LLMs are not automatically HIPAA compliant applies that distinction to health information.
Convert every requirement into an acceptance test
Start with the data policy, not a vendor questionnaire. State which data classes the system may receive, where they may travel, which uses are permitted and when every copy must be removed. A policy that says only "use a private LLM" cannot be tested.
Turn each row of the constraint map into a pass-or-fail condition. For retention, name every covered artifact and require the provider to disclose standard and exceptional windows. For training use, specify whether the prohibition covers prompts, outputs, uploaded files, feedback and derived classifications. For egress, define the permitted destinations and test outbound connections from the deployed environment.
Inspect logs using representative synthetic data, meaning invented records that contain no real personal or confidential information. Confirm whether prompt text, retrieved documents or generated output appears in application, gateway, inference and monitoring records. Configuration alone can miss logging performed by another component.
Test deletion by assigning a traceable identifier to the synthetic request. Submit the deletion operation, then search each known store for the identifier. This verifies the implemented workflow across stores under the operator's control. It does not prove that an external provider has erased an inaccessible copy, so the final assessment should distinguish observed evidence from contractual evidence.
Consider an engineer running this test who submits a synthetic support ticket, deletes it through the documented workflow, and then finds the same ticket text sitting in a nightly backup snapshot that no one had listed as a store to check. The deletion procedure worked exactly as written; it had simply never been told the backup existed.
For systems processing personal data under the General Data Protection Regulation, purpose and deletion cannot be reduced to hosting location. Personal data must be collected for specified, explicit and legitimate purposes and must not be reused incompatibly with those purposes. 5 The same regulation provides a right to erasure when an applicable ground exists, which makes the reach of the deletion workflow a design requirement rather than a support preference. 5
Encryption keys and deletion promises have hard limits
A customer-managed encryption key is a key controlled through the customer's permissions rather than solely through the provider's defaults. It can restrict access to stored ciphertext, meaning data rendered unreadable without decryption.
It does not prevent a managed service from decrypting data when authorised to process a request. Google Cloud documents that a service agent, an automated identity used by the managed service, performs encryption and decryption with customer-managed keys. 6 Key ownership therefore supports the at-rest control, while runtime access still requires a separate assessment.
Deletion becomes harder after information has been used to train a model. Model weights are the learned numerical parameters that shape its outputs. Blocking a particular response with an output filter does not remove information from those parameters. The UK Information Commissioner's Office states that output filters may help implement information rights but do not remove the underlying data from the model. 7
A remote service can provide terms, audit reports and deletion confirmations, but the customer cannot inspect an absent record inside systems it does not control. Where that evidentiary gap is unacceptable, the architecture must prevent the sensitive field from being transmitted. Redaction before the request, local processing or exclusion of the use case changes the privacy claim from "the recipient deleted it" to "the recipient never received it."