The answer
Your records can remain on infrastructure you control. A language model generates text from its instructions and available context; hosting it locally does not itself establish permissions or prevent other components from sending data outside. Map the complete data path before deciding the deployment boundary.
What follows is the bill, and the proof that the label is true.
What the rules actually require
European data-protection law does not order you to keep files in the cupboard. It says a transfer of personal data to another country may happen only if the conditions in that chapter are met (1, accessed 2026-09-09). Leaving is allowed when those conditions hold, so staying is a choice you make, not a ceiling the rule sets.
What hosted terms actually say
A published set of business terms for a hosted language model says customer data is encrypted in transit between you and the provider (2, accessed 2026-09-09). "In transit" means the file left. Those terms also offer storage at rest in named regions, and a region is still not your cupboard.
If the leasing manager pastes a rider into a hosted tool, the rider has gone. If the front desk at nine in the evening uses a personal account, it has gone with no contract. The hosted path can be lawful when the transfer conditions are met, but it is not in-house. Do not treat a regional store as a building you can walk into.
What keeping it here costs
Size the hardware against the model, peak requests and response-time requirement before asking for a quote. Include memory, storage, backup capacity and support. A generic rack-server starting price is not a minimum cost for running an LLM. Use the hardware sizing guide to define the workload first.
For an illustrative power estimate, assume a machine draws 1,000 watts continuously for 30 days. It uses 720 kilowatt-hours. At an assumed tariff of $0.1419 per kilowatt-hour, electricity costs $102.17 before cooling and other facility overhead. Replace both the draw and tariff with your own measurements.
Budget an operating owner for patching, backups, outbound-traffic monitoring and incident response. Agree cover outside normal hours. These responsibilities consume time even when an existing employee takes them on.
In-house is a claim, not a label
A sticker on the rack is not proof. Proof looks like a log that shows no record left, a blocked path out of the building, and a test that a lease rider never appears on a wire you do not control. Staff laptops, cloud backups, and a remote-support session can move a file while the box stays in the room. On-premise describes where the system runs. Backups, updates and support connections need separate controls before you can claim that protected records stay inside.
Test outbound paths using synthetic records and inspect network configuration, traffic logs, backup destinations and support access. Keep the results with the release record. Include ERP integrations in the test: a locally hosted model may still connect to a remotely hosted business system.
What this does not solve
Keeping records in the building does not make the draft right. A clerk still matches the goods receipt to the order. It does not buy cooling, a spare machine, or a person on call after the front desk closes, and it does not sign off security or procurement. Those sit with other desks.
Before launch, assign an operating owner, access controls, power and cooling, backup arrangements and time to review logs. Include staff devices and physical copies in the handling rules; local inference does not control them.
A list for the next operations meeting
No engineer is required. Bring this table.
| Check | Pass if |
|---|---|
| Who owns the box on the org chart | A name, not a team |
| Where it sits | A room you can walk to this week |
| Who pays the power | A cost centre you control |
| Who reads the outbound logs | A person who can do it this week |
| What files it may see | Named: lease rider, goods receipt, evening spreadsheet |
| What would count as a leak | A record on a wire you do not own |
| Who approves the data-handling boundary | The responsible security and data owners |
If a row is blank, the label is not earned. Run the list before you ask for a purchase. The purchase is the easy part. The dated log is the job.