When self-hosting an LLM is actually cheaper than an API
The question is usually asked as if there were one answer. There is not, because self-hosting does not compete with 'the API'. It competes with a specific price per million tokens, and those prices differ by a factor of forty between a frontier model and a hosted open-weight one.
By Evdokiia Petrovskaia, Director, Dubir GroupPrices checked
Short version: if you are replacing a frontier closed model, a single GPU pays for itself somewhere around 15 million tokens a month. If you are replacing a hosted open-weight endpoint, it essentially never does. Decide which of those two you are actually doing before pricing hardware.
What self-hosting costs per month
Four lines, and only the first is obvious.
- Depreciation. The purchase price spread over its useful life. A $2,900 RTX 4090 over 36 months is about $81 a month. Treating the card as free because you already own it is the single most common error in these comparisons.
- Electricity. A 4090 draws 450 W flat out, but a real workload averages far less. At a 35% duty cycle and €0.28 per kWh, about $32 a month.
- Operations. Patching, monitoring, restarting, upgrading a runtime that ships breaking changes monthly. 20% of hardware plus power is a deliberately conservative figure and it is still $23 a month.
- Idle time. Not a line item but the real economics: the hardware bill is the same whether the GPU is saturated or asleep, while the API bill goes to zero on a quiet week.
Total for the 4090 example: about $135 a month, of which $55 is recurring once the card is written off.
The breakeven depends on what you are replacing
The same $135 a month, compared against different APIs at 20 million input and 5 million output tokens per month:
| Replacing | API cost/month | Breakeven | Payback |
|---|---|---|---|
| Frontier closed model | $225 | 15M tokens/mo | 17 months |
| Mid-tier closed model | $100 | 34M tokens/mo | 64 months |
| Qwen3 30B-A3B, hosted | $5 | 690M tokens/mo | never |
That last row is the one worth sitting with. Hosted open-weight inference is so cheap that buying a graphics card to avoid a five-dollar bill takes 690 million tokens a month to justify, and at that point you are running a serving operation, not saving money on a side project.
Bigger hardware does not fix it
A $4,500 RTX 5090 at 50 million input and 12 million output tokens a month costs about $199 a month self-hosted against $244 on a mid-tier API: 26 months to payback, with a breakeven of about 51 million tokens a month. More capable hardware raises both the cost and the breakeven roughly in step. The ratio is set by the API price you are escaping, not by the card.
Capacity is not the constraint
One card generating 300 tokens a second around the clock produces about 788 million output tokens a month. At 60 tokens a second, a dense model on a Mac, it is still 158 million. For almost every internal tool, one GPU has more throughput than the organisation has demand, which means you are buying idle capacity and the utilisation question matters more than the speed question.
The reasons that actually hold up
Cost is the weakest argument for self-hosting and the one most often given. The durable ones:
- The data cannot leave. Regulated industries, client confidentiality, personal data a processor agreement will not cover. This is the reason we deploy models on client hardware, and it does not depend on token volume at all.
- Predictable cost. A fixed monthly number beats a variable one for some finance departments even when the variable one is lower on average.
- No dependency on a vendor's roadmap. Models you host are not deprecated, repriced or rate-limited without your consent.
- Latency and offline operation. A local model answers without a round trip, and keeps answering when the connection does not.
Work out your own number
Put your actual monthly token volume, your actual electricity price and the actual card you are considering into the self-hosted vs API calculator. Every figure above came out of it. If you have not yet decided which card, 4090 vs 5090 vs Mac compares the three on the two specifications that decide it.
Questions, answered
At what volume does self-hosting an LLM become cheaper?
There is no single number, because it depends on the API price you are comparing against. Replacing a frontier closed model, an RTX 4090 breaks even at roughly 15 million tokens a month. Replacing a mid-tier closed model, about 34 million. Replacing a cheap hosted open-weight endpoint at $0.12 per million input tokens, roughly 690 million a month, which is more than most companies will ever send.
How long does a GPU take to pay for itself?
In our worked example a $2,900 RTX 4090 replacing a frontier API at 25 million tokens a month pays back in about 17 months. Against a mid-tier model at the same volume, about 64 months, which is longer than the hardware's useful life. Against hosted open-weight models it does not pay back at all.
What costs am I forgetting when I self-host?
Electricity, which is small but not zero: a 4090 at 35% duty cycle costs about $32 a month at €0.28/kWh. Depreciation, which is the largest line and which people leave out entirely by treating the purchase as sunk. And operations: someone has to patch, monitor and restart it, and 20% of the hardware and power cost is a conservative allowance for that.
Does self-hosting mean I get the same quality?
Not automatically. The honest comparison is against hosting the same open-weight model through an API, and there the economics rarely favour buying hardware. The comparison that does favour it is against a closed frontier model, and that is a quality trade as well as a cost one. You are asking whether an open model is good enough for your task.
How many tokens can one GPU actually produce?
More than most people need. At 300 tokens a second running continuously, one card produces about 788 million output tokens a month. Capacity is rarely the binding constraint for an internal tool; the constraint is that you are paying for the card whether or not it is busy.
So when is self-hosting the right call?
When the data is not allowed to leave your infrastructure, when you need predictable cost rather than lowest cost, when latency to your own hardware matters, or when your volume against a frontier API is genuinely large. Privacy and regulation are better reasons than price, and they are the ones that hold up.
We build the software behind numbers like these
Dubir Group is a technology company in Paphos, Cyprus. Payroll systems, internal tooling, and AI that runs on your own hardware when the data cannot leave.