Private LLMs and Self Hosted AI: Costs, Models and Data Residency for Enterprises

Every enterprise AI project reaches the same meeting: someone from legal or security asks where the data goes when an employee types a question into the new assistant. The answer decides whether you can use a hosted model as it is, need a private LLM deployment in your own cloud, or must run a self hosted LLM on hardware you control. Get it wrong one way and you block a useful project; get it wrong the other way and you spend six figures on GPUs you did not need.

This guide gives you 2026 GPU prices with sources, a monthly cost comparison you can rerun, current open weight licences, and the residency rules that matter in the United States, United Kingdom and United Arab Emirates. We design these setups as part of our AI architecture and engineering work, and most of them end up powering enterprise RAG and AI search over company documents.

What is a private LLM, and how is it different from a self hosted LLM or sovereign AI?

A private LLM processes your prompts under enterprise terms, isolated from other customers and never used for training; a self hosted LLM goes further, because you run the model weights yourself.

• Private LLM. Prompts stay under enterprise terms, unused for training, ideally reachable only through your private network. Includes hosted models inside your cloud account.
• Self hosted LLM. You run open weight model files on GPUs in your cloud account or data centre, served by an inference engine such as vLLM or SGLang.
• On premise LLM. A self hosted model on hardware in your own building or a colocation facility, sometimes air gapped from the internet.
• Sovereign AI. Data, compute and, in stricter readings, the model and its operators all sit under one country’s legal control.

The real question is who may see your data, where it may be processed, and who can switch the system off.

When do you actually need a private LLM or self hosted AI?

You need a private deployment whenever regulated or confidential data will reach the model, and you need self hosting only when even an enterprise cloud API cannot meet your residency, contract or cost constraints. In our experience the second case is much rarer than people expect.

Signals that a private deployment in your cloud is enough

• Your policy requires enterprise terms, no training on your data and a data processing agreement.
• Data must stay in a region, such as the UK, EU or US, where the models you need are offered regionally.

Signals that you may need to self host

• A law requires data to stay in a country where no suitable hosted model is processed locally, as with much UAE health data.
• Client contracts, common in law firms and defence supply chains, forbid any third party model provider.
• You need an air gapped system, or a model version frozen for years for audit reasons.
• You run very high, steady volume where dedicated GPUs cost less than per token pricing.

When not to self host

Do not self host because it feels safer or to save money at low volume. If nobody will own patching and on call, or your workload needs the best available reasoning, a hosted model in your own tenancy is the better answer. Self hosting without operations is less private, not more.

Four ways to deploy a private LLM

Enterprises have four realistic options. Each moves more control, and more work, onto your team.

1. Hosted frontier models inside your cloud tenancy

Azure OpenAI in Microsoft Foundry, Amazon Bedrock and Google Vertex AI let you call leading models, including OpenAI’s GPT models and Anthropic’s Claude, through your own cloud account with private endpoints, identity controls and terms that exclude training on your prompts. Check the deployment type. AWS explains that with cross region inference from its UAE and Bahrain regions, invocation logs and knowledge bases stay in the source region while the request itself may be served elsewhere (AWS, February 2026). Geography pinned processing often costs more: Anthropic charges 1.1 times its standard token prices for US only inference (Anthropic pricing).

2. Managed open weight models

The same platforms host open weight models such as Mistral, Qwen and gpt oss as serverless endpoints. You avoid GPU operations while choosing a model you could later move in house.

3. Open weight models on GPUs in your cloud

You rent GPU instances in your AWS, Azure or Google Cloud account, or from a GPU cloud, and serve the model yourself. vLLM is the common default, with SGLang a strong alternative. Hugging Face Text Generation Inference, once a popular choice, was put into maintenance mode and its repository archived in March 2026, with the maintainers recommending vLLM or SGLang instead (GitHub). Data never leaves your network, and you choose the region, subject to which GPU types that region offers.

4. On premises GPUs

You buy or lease GPU servers and run the same stack in your own data centre. It gives the most control and suits air gapped environments, but brings capital cost, hardware lead times, power and cooling, and round the clock operations staff.

Option	Data control	Model quality	Operations effort	Cost shape
Hosted frontier model in your tenancy	High, under enterprise terms	Highest available	Low	Per token
Managed open weight endpoint	High	Good to very good	Low	Per token
Open weight on your cloud GPUs	Very high	Good to very good	Medium to high	Per GPU hour
On premises GPUs	Full	Good to very good	High	Capital plus running

Which open weight models can you self host in 2026, and under which licences?

The best model for a self hosted LLM is the smallest one that passes your own evaluation set, under a licence your lawyers accept. In 2026 the strongest open weight releases come from Alibaba, Mistral, Google, OpenAI and DeepSeek, and most use permissive Apache 2.0 or MIT licences. Meta, once the default, has slowed: Llama 4 dates from April 2025, and its newer Muse Spark model, released in April 2026, was not published as open weights (Pebblous, 2026).

Model family	Current open releases	Licence	Notes for enterprises
Qwen by Alibaba	Qwen3.5 and Qwen3.6, including a 27B multimodal model	Apache 2.0	Many sizes, long context; a good default for document work
Mistral	Mistral Large 3 (675B total, 41B active) and Ministral 3 at 3B, 8B and 14B	Apache 2.0	European provider; some other Mistral models use more restrictive terms
Gemma by Google	Gemma 4, released April 2026	Apache 2.0	First Gemma generation under Apache 2.0; earlier versions used custom terms
gpt oss by OpenAI	gpt oss 20b and 120b	Apache 2.0	OpenAI says the 120b model runs on a single 80 GB GPU
DeepSeek	DeepSeek V4, up to about 1.6 trillion total parameters	MIT	Very capable but needs a multi GPU node; check your policy on model origin
Llama by Meta	Llama 4 Scout and Maverick	Llama 4 Community License	Separate licence above 700 million monthly users; EU based companies are excluded from the multimodal models
Arabic models	Jais 2 (Inception, MBZUAI, Cerebras) and Falcon H1 Arabic (TII)	Check each release	Strong Arabic quality for UAE and Gulf projects that must stay in country

Licence details from IntuitionLabs (September 2026), Qwen on Hugging Face, Google Open Source Blog, OpenAI and MBZUAI. Licences differ between sizes in the same family, so read the licence of the exact checkpoint you deploy.

Should you fine tune a private model on your documents? Usually retrieval over current documents works better and costs less; keep fine tuning for format, tone or narrow classification.

How much GPU do you need for a self hosted LLM?

A rough sizing rule is that model weights need about 2 bytes of GPU memory per parameter at 16 bit precision, 1 byte at 8 bit and about half a byte at 4 bit, plus 20% to 50% headroom for the context cache that grows with users and document length. This is our planning rule of thumb, and a load test on your prompts always has the final word.

Model size	Weights at 8 bit	Typical starting hardware	Good for
3B to 14B, for example Ministral 3	About 3 to 14 GB	One mid range data centre GPU	Classification, routing, extraction, redaction
About 27B, for example Qwen3.6 27B	About 27 GB	One 80 GB GPU such as an NVIDIA H100	Internal assistants and document question answering
About 120B with 4 bit weights, for example gpt oss 120b	Fits one 80 GB GPU	One H100, two for redundancy	Harder reasoning and tool use
Several hundred billion and up, for example Mistral Large 3 or DeepSeek V4	Hundreds of GB	A multi GPU node of H100, H200 or B200 cards	Near frontier quality where budget allows

2026 rental prices give a sense of scale. An AWS p5.4xlarge with one H100 80 GB lists at $6.88 an hour on demand in US East and about $2.97 an hour on a three year reservation (Vantage). Lambda lists an H100 at $3.99 and a B200 at $6.69 per GPU hour on demand, with no egress fees (Lambda). Always plan at least two GPUs for a production service so one failure does not take the assistant down.

How much does a private LLM cost per month? A worked example

At moderate volume, a hosted model in your tenancy is usually cheaper than self hosting; self hosting starts to win only at high, steady volume with committed GPU pricing. Here is an example scenario with every assumption stated, so you can rerun it with your own numbers.

Example scenario. A company with 2,000 staff runs an internal assistant over its documents. It handles 20,000 requests per working day, 22 working days a month, so 440,000 requests a month. Each request sends about 3,000 input tokens (the question plus retrieved passages) and gets about 400 output tokens back. That is 1.32 billion input tokens and 176 million output tokens a month.

Setup	Monthly compute or usage	Operations	Approximate monthly total
A. Hosted model in your tenancy at $2 input and $10 output per million tokens	$2,640 input plus $1,760 output	Minimal	About $4,400 (about $4,840 with US only processing at 1.1 times)
B. Self hosted open weight model on two AWS H100 instances, on demand	2 x $6.88 per hour x 730 hours	About $4,000 of engineering time	About $14,000
C. Same as B on a GPU cloud at $3.99 per GPU hour	2 x $3.99 x 730 hours	About $4,000	About $9,800
D. Same as B with three year reserved pricing	2 x $2.97 x 730 hours	About $4,000	About $8,300
E. Five times the volume: hosted model	About $22,000 usage	Minimal	About $22,000
F. Five times the volume: four reserved H100 instances	About $8,700	About $4,000	About $12,700

How we got these numbers. The token price in row A matches Claude Sonnet 5.5 on Anthropic’s price list as of October 2026; your chosen model and cloud may differ. GPU prices are the list prices quoted above. Two instances give redundancy. The operations line assumes a part time engineer for patching, upgrades, monitoring and on call, similar to an engineering retainer from $4,000 per month. Capacity per GPU is our planning assumption for a mid size model served by vLLM and must be confirmed by a load test.

At moderate volume the hosted model costs roughly half of the cheapest self hosted setup. Self hosting wins at high, steady volume, but only with committed capacity. Prices also vary by region, so price the region you will actually use. For the wider budget picture, see our guide to AI development cost in 2026.

Key takeaway: choose self hosting for control, not to save money, unless your volume is high and predictable enough to justify a multi year GPU commitment.

Security for self hosted AI: what changes when you run the model?

Self hosting removes the model provider from your threat model but adds every job the provider used to do.

• Model supply chain. Download weights only from the publisher’s official repository, verify checksums, and prefer the safetensors format over formats that can execute code on load.
• Network isolation. Private subnets with no outbound internet access, reachable only through your gateway.
• Patching. Inference engines such as vLLM ship security fixes regularly; someone must own upgrades and testing.
• Logs are sensitive data. Prompts and outputs contain the same confidential material as the source documents. Encrypt them, set retention limits and restrict access.
• Permissions in retrieval. A private model can still show a document to the wrong employee if the search index ignores access rights. This is the most common flaw we find in internal AI search projects.
• Prompt injection and output handling. The OWASP Top 10 for LLM Applications applies whether the model is hosted or not. Guard models can screen inputs and outputs.

Private LLM data residency rules in the US, UK, UAE and EU

The strictest residency requirements usually come from sector rules and contracts rather than general privacy law.

United States

The US has no general rule that business data must stay in the country, and HIPAA does not require US hosting, although it does require a business associate agreement with any vendor that handles protected health information. The main federal restriction is the Justice Department’s Data Security Program, 28 CFR Part 202, in effect since April 2025, which limits access to bulk US sensitive personal data by countries of concern and covered persons (eCFR). In practice, a hosted model pinned to US processing, such as a US data zone deployment, meets most private sector policies. See our US page for how we work with American teams.

United Kingdom

UK GDPR, as amended by the Data (Use and Access) Act 2025, does not require data to stay in the UK, but many public sector, health and financial buyers set UK hosting as policy. Azure UK South, AWS Europe (London) and Google Cloud London all offer hosted models, so a private deployment in your tenancy usually satisfies UK requirements. Our UK buyer’s guide covers the wider rules.

United Arab Emirates

The PDPL, Federal Decree Law No. 45 of 2021, allows transfers abroad to adequate countries and in defined cases, but its Executive Regulations had still not been issued as of September 2026, so detailed transfer rules are pending (ITSecNow, 2026). DIFC and ADGM have their own data protection laws. Health data is stricter: Article 13 of Federal Law No. 2 of 2019 keeps it in the UAE except where ministerial rules allow. UAE sovereign setups use Azure UAE North, the AWS Middle East (UAE) region, local sovereign cloud offerings, or self hosted models such as Jais 2. Our guide to AI development in Dubai and the UAE and our UAE page cover local buying questions.

European Union

GDPR governs personal data, and the EU AI Act adds duties by risk level. The AI Omnibus, Regulation (EU) 2026/1744, entered into force on 27 July 2026 and moved obligations for standalone high risk systems to 2 December 2027 and for AI in regulated products to 2 August 2028 (White & Case). For EU sovereignty projects, check model licences as closely as hosting, because Llama 4’s multimodal models exclude EU based companies while Apache 2.0 models from Mistral, Qwen and Gemma do not.

What trade offs do you accept with a self hosted LLM?

Self hosting buys control at the price of model quality, speed of improvement and operating effort. Be honest about each before you commit.

• Quality gap. The strongest open weight models are good, but the best hosted frontier models still lead on hard reasoning, long documents and complex tool use. Measure the gap on your tasks.
• Slower upgrades. A hosted model improves when the provider ships a new version. Upgrading a self hosted model means new benchmarks, new capacity plans and a deployment.
• Capacity planning. Hosted APIs absorb traffic spikes. Your own GPUs queue requests when busy, so you size for peak or accept slower answers.
• People. Someone must be on call for the inference cluster.

The best designs keep options open. Route every model call through a gateway in your own code, so moving a task between a hosted model and a self hosted one is a configuration change followed by a test run. Our AI integration projects build that gateway by default.

Which private LLM option fits your organisation?

Most organisations land on a hosted model in their own tenancy, and a minority genuinely need a self hosted LLM. The table gives our honest default by buyer type; your contracts and data classification can override it.

Buyer	Our default recommendation	Why
Company under 200 staff, no regulated data	Enterprise plan of a hosted model, or an API in your cloud	Self hosting costs more than it saves at this size
Mid market company with client confidential data	Hosted model in your tenancy, private endpoints, regional processing	Meets most contracts with little operations work
US healthcare provider	Hosted model under a BAA in your tenancy; self host only for special cases	HIPAA needs a BAA and safeguards, not US only hardware
UK public sector or NHS supplier	UK region deployment in your tenancy	UK hosting policy is met without running GPUs
UAE health or government data	Self hosted open weight or Arabic model in a UAE region or on premises	In country processing is often mandatory
Law firm with no third party AI clauses	Self hosted model in your cloud account	The contract, not the law, forces it
Product with very high, steady volume	Price both; reserved GPUs may win	Per token costs scale with every request

The same logic applies to narrower workloads. Our guide to AI invoice processing shows how a finance team can keep supplier data in its own cloud account without self hosting a large model, and our build vs buy guide covers the general decision.

A practical six step path to a private LLM

This sequence gets most enterprises to a safe, measured decision in 4 to 8 weeks without buying hardware first.

1. Classify the data. List the data types the assistant will see and the laws and contracts that apply to each.
2. Set residency and provider rules. Decide which regions, providers and model origins are acceptable, in writing, with legal and security.
3. Build an evaluation set. Collect 100 to 300 real questions with approved answers, including your hardest cases and, where relevant, Arabic.
4. Test a hosted model in your tenancy first. It is the fastest way to learn what good looks like on your data.
5. Test two or three open weight models. Serve them with vLLM in your account and score them on the same set.
6. Price both at your real volume. Include GPUs, reserved pricing, operations time and the cost of the quality gap.

How Vrisic can help

We help enterprises choose between hosted, private and self hosted models, then build the system on top. An architecture review with model benchmarks on your data typically costs $8,000 to $20,000 (£6,400 to £16,000, AED 29,000 to AED 73,000) over 2 to 4 weeks, and department search with permissions typically $40,000 to $100,000 (£32,000 to £80,000, AED 147,000 to AED 367,000). These are typical ranges; you get one fixed price in writing after a free scoping call. Everything runs in your cloud account, you own the code, prompts and evaluation sets, and hosting and model usage are billed at cost. If you are weighing a private LLM for healthcare data, legal documents or UAE resident data, we will tell you which option fits, including when the answer is simply to configure the cloud you already have.

The quickest way to start is our AI Pilot: $1,500 (£1,200, AED 5,500), 10 business days, fully credited if you continue within 60 days. You can also message us on WhatsApp at +91 63777 67206 or book a scoping call.

Sources

• Anthropic, Claude API pricing
• Vantage, AWS p5.4xlarge pricing
• Lambda, GPU cloud pricing
• AWS, Bedrock global cross region inference in the Middle East, February 2026
• vLLM documentation and Hugging Face Text Generation Inference repository
• IntuitionLabs, open weight model licences, September 2026
• Qwen3.6 27B model card, Google, Gemma 4 under Apache 2.0, OpenAI, Introducing gpt oss
• Pebblous, Meta drops open weights for Muse Spark, 2026
• MBZUAI, Jais 2 release, December 2025
• eCFR, 28 CFR Part 202
• ITSecNow, UAE PDPL Executive Regulations status, September 2026, and UAE Government portal, data protection laws
• White & Case, EU AI Omnibus enters into force, July 2026

Prices and model availability change often; figures here are examples as of October 2026, not quotes. This guide is not legal advice.