When self-hosting an LLM is actually cheaper than an API
A GPU you buy competes with a specific price per million tokens, and API prices differ by a factor of forty between a frontier model and a hosted open-weight one. So whether self-hosting is cheaper depends mostly on which API you'd be replacing.
By Evdokiia Petrovskaia, Director, Dubir GroupPrices checked
If you're replacing a frontier closed model, a single GPU pays for itself somewhere around 19 million tokens a month. If you're replacing a hosted open-weight endpoint it pretty much never does, so work out which of the two you're doing before you price any hardware.
What self-hosting costs per month
Four lines, and only the first is obvious.
- Depreciation is the purchase price spread over the card's useful life, so a $2,900 RTX 4090 over 36 months is about $81 a month. The card you already own still costs this, and leaving it out because the purchase is sunk is the error we see most in these comparisons.
- A 4090 draws 450 W flat out, and a real workload averages far less. At a 35% duty cycle and €0.28 per kWh, the electricity is about $32 a month.
- Then someone has to patch it, watch it, restart it and upgrade a runtime that ships breaking changes monthly, and whoever does it gets paid. We charge 20% of hardware plus power for that, a low estimate, and it still comes to $23 a month.
- Idle time isn't a line item at all, which is why it gets missed. The hardware bill is identical whether the GPU is saturated or asleep, while the API bill goes to zero on a quiet week.
Total for the 4090 example: about $135 a month, of which $55 is recurring once the card is written off.
The breakeven depends on what you are replacing
Here is the same $135 a month against different APIs, at 20 million input and 5 million output tokens per month:
| Replacing | API cost/month | Breakeven | Payback |
|---|---|---|---|
| Frontier closed model | $180 | 19M tokens/mo | 23 months |
| Mid-tier closed model | $90 | 38M tokens/mo | 82 months |
| Qwen3.6 35B-A3B, hosted | $8 | 420M tokens/mo | never |
Look at the last row. Hosted open-weight inference is so cheap that a graphics card bought to avoid an eight-dollar bill needs 420 million tokens a month to justify it, and at that volume you're running a serving operation, which is a different project.
Bigger hardware doesn't fix it
A $5,000 RTX 5090 at 50 million input and 12 million output tokens a month costs about $216 self-hosted against $220 on a mid-tier API. That is 35 months to payback, breakeven around 61 million tokens a month, and the card is close to the end of a three-year write-off by the time it has earned itself back. Bigger hardware raises the cost and the breakeven together, because the ratio comes from the API price you're moving away from.
Capacity is rarely the problem
One card generating 300 tokens a second around the clock produces about 788 million output tokens a month. At 60 tokens a second, a dense model on a Mac, it is still 158 million. For almost every internal tool one GPU has more throughput than the organisation has demand, so most of what you buy is idle time. The question to ask in a procurement meeting is how many hours a day anyone will be waiting on the card, and its tokens-per-second figure barely comes into it.
Reasons that hold up
Cost is the weakest argument for self-hosting and also the one people give most often. These hold up better, and none of them fit in a spreadsheet:
- The data can't leave: regulated industries, client confidentiality, personal data a processor agreement won't cover. That's why we deploy models on client hardware, and token volume has nothing to do with it.
- Some finance departments prefer a fixed monthly number to a variable one, even if the variable one is lower on average.
- Models you host aren't deprecated, repriced or rate-limited without your consent.
- A local model answers without a round trip and keeps answering when the connection drops.
Work out your own number
Put your actual monthly token volume, your actual electricity price and the actual card you are considering into the self-hosted vs API calculator. Every figure above came out of it. If you haven't picked a card yet, 4090 vs 5090 vs Mac compares the three on the two specifications that decide it.
Questions, answered
At what volume does self-hosting an LLM become cheaper?
It depends on the API price you're comparing against. Replacing a frontier closed model, an RTX 4090 breaks even at roughly 19 million tokens a month; replacing a mid-tier closed model, about 38 million. Against a cheap hosted open-weight endpoint at $0.15 per million input tokens it takes roughly 420 million a month, more than most companies will ever send.
How long does a GPU take to pay for itself?
In our worked example a $2,900 RTX 4090 replacing a frontier API at 25 million tokens a month pays back in about 23 months. Against a mid-tier model at the same volume, about 82 months, which is longer than the hardware's useful life. Against hosted open-weight models it doesn't pay back at all.
What costs am I forgetting when I self-host?
Electricity is small but real: a 4090 at 35% duty cycle costs about $32 a month at €0.28/kWh. Depreciation is the biggest line, and people leave it out completely when they treat the purchase as sunk. Then operations, since someone has to patch, monitor and restart the box, and we allow 20% of the hardware and power cost for that (a low estimate).
Does self-hosting mean I get the same quality?
That depends on what you compare it with. Against the same open-weight model hosted through an API the quality is the same, and the economics rarely favour buying hardware. Against a closed frontier model, where the numbers do favour it, you're trading quality as well as cost, so what you're really asking is whether an open model is good enough for your task.
How many tokens can one GPU actually produce?
At 300 tokens a second running continuously, one card produces about 788 million output tokens a month, more than most people need. For an internal tool the limit is rarely capacity. It's that you pay for the card whether or not it's busy.
So when is self-hosting the right call?
When the data isn't allowed to leave your infrastructure, when you'd take a predictable bill over the lowest one, when latency to your own hardware matters, or when your volume against a frontier API is really large. Privacy and regulation are stronger reasons than price.
Private AI, from the spec to the running box
Dubir Group deploys open models on client hardware from Paphos, Cyprus, for the cases where the data is not allowed to reach a third-party API. We size the machine, install it and keep it answering.