Private LLM deployment: open models on your own servers, so the data stays in-house.
Private AI for business, on-premise LLM, self-hosted LLM
We deploy open language models on hardware you own or rent, in your office or your own cloud account, for work where prompts and documents can't go to a third-party API. We size the machine, install the model and the serving stack, connect it to your tools and keep it answering.
By Evdokiia Petrovskaia, Director, Dubir GroupCopy checked
What a private LLM deployment includes
| Part | What we do | What you get |
|---|---|---|
| Sizing | We match the model to the job and to the memory, with the arithmetic our hardware calculator uses: weights plus KV cache plus overhead. | A hardware list with prices before you buy anything. |
| Hardware | A workstation with one or two GPUs, a Mac with a lot of unified memory, GPU servers in your own cloud account, or verifiably private inference in confidential-computing enclaves (Intel TDX/SGX) when buying a box isn't the right move. | Hardware or a cloud account in your name. We never hold the disks. |
| Model and serving | llama.cpp or Ollama for a handful of users, vLLM when twenty people use it at once, with an OpenAI-compatible API either way. | Your existing tools and scripts point at your own server and keep working. |
| Connections | A chat for staff, search over your documents, and the automations that call the model. | Something people use on day one. |
| Running it | Monitoring, fixes, and a model swap when a better open model comes out. | A monthly fee you can cancel in any month. |
Which models, on which hardware
Our LLM hardware calculator tracks 22 open models, from 2B models that run on a laptop up to Qwen3.8 2.4T, each with its Terminal-Bench 2.1 agent score next to the memory it needs. A 24GB card like the RTX 4090 runs 17 of them at 4-bit and 8K context. A 128GB Mac loads the 120B models neither the 4090 nor the 5090 can, at roughly a third of the speed, and the 5090 is about 1.78 times a 4090 (the exact ratio of their memory bandwidth). Unified memory gives a model about 75% of the total, a dedicated GPU about 90%, which the VRAM guide works through model by model.
What we measured on our own machine
We run open models for our own work as well. Qwen3.8 27B at Q4_K_M (19GB) on an M1 Max with 64GB decodes 11.8 tokens a second plain and 16.4 to 17.7 with DFlash 2 speculative decoding, using 22.4GB of memory with an 8K context. It reads prompts at about 69 tokens a second, so an 8,000-token prompt waits around two minutes before the first word comes out. We used it to classify a batch of keyword research offline, 49 of 49 answers valid at 2.9 seconds each. Two things cost us a full run each before that: Qwen3.8 thinks by default and hands back an empty answer unless thinking is switched off per request, and JSON only comes out clean every time when a grammar forces it. Both checks are now part of every deployment we do.
Why companies go private
Usually because of the data: client files, health records, contracts under NDA, or a regulator who wants to know where every prompt went. Cost can count too, an RTX 4090 pays for itself against a frontier closed model from about 19 million tokens a month, and the self-hosted vs API calculator does that sum for your own volume, electricity and ops included. Once the server is running, the automations that would otherwise call OpenAI can call it instead, at no cost per token.
What it costs
The hardware is yours and priced in the sizing step, from one workstation to a rack of GPU servers. Setup is quoted once we know the model, the number of users and what it connects to, and running it afterwards is a monthly fee you can cancel in any month. The first call is free, and you'll leave it with a machine spec!
Questions, answered
What is private LLM deployment?
Running a large language model on hardware you control, in your office or in your own cloud account, so prompts, documents and answers never pass through OpenAI, Anthropic or anyone else. The model is open-weight, downloaded once, and the server can run with no internet connection at all.
Is a private LLM as good as ChatGPT?
For drafting, summarizing, classifying and answering from your own documents, a 27B to 120B open model does the job. The hosted frontier models still lead on the hardest reasoning, so some teams use both and send only the sensitive work to the private server.
What hardware do we need?
It depends on the model and how many people use it at once. A 24GB card like the RTX 4090 runs 17 of the 22 open models our calculator tracks at 4-bit and 8K context, and a 128GB Mac or a DGX Spark loads the 120B models a single Nvidia card can't. The free hardware calculator gives you the answer for your own case in a minute.
Can you deploy it in our own cloud account?
Yes, on GPU instances in your AWS, Azure or Google Cloud account, or with a European host if you need the data kept in the EU. The account and the bill stay yours.
Ollama, LM Studio or vLLM?
Ollama or llama.cpp for one person or a small team, LM Studio when someone wants a desktop app, vLLM when many people hit the server at once (it has no Apple Silicon support). Our comparison guide has the details and the numbers.
How much does private LLM deployment cost?
The hardware is yours and priced in the sizing step. Setup is quoted once we know the model, the number of users and what it connects to, and running it afterwards is a monthly fee you can cancel in any month.
Ask before you buy the hardware
Tell us what the model has to do and for how many people, and we'll open the calculator together and price the machine on the call.