Self-hosted LLM vs API: where you break even
Per-token pricing looks cheap until the volume arrives. Put in what you send each month and what a GPU would cost you to buy, power and babysit, and see which side of the line you are on, and how many tokens a month it takes to cross it.
Commercial API
$450/ month
- Input, 40 million tokens
- $200
- Output, 10 million tokens
- $250
- Per year
- $5,400
Your own hardware
$195/ month
- Hardware, over 36 months
- $80.56
- Electricity
- $82.13
- Ops at 20%
- $32.54
- After it is paid off
- $115 / month
The verdict
Owning the hardware is cheaper at your volume: $195 a month against $450 on the API, saving $255 a month, or $3,057 a year.
The two lines cross at 22 million tokens a month at your input/output mix. Below that the API wins; above it the hardware does. At today's volume the hardware pays for itself in 8.6 months.
Capacity check: that box runs Qwen3.6 35B-A3B (MoE) at 4-bit and could generate roughly 874 million output tokens a month on a single stream at full duty. Serving several requests at once multiplies that several times over.
Prices in USD. Power assumes the machine is on 730 hours a month. This compares cost only; it does not price the part where your data stays in your building.
What each side of the comparison contains
The arithmetic behind each field above, folded so you can open just the part you are questioning.
The API side
One multiplication per direction: input tokens times the input price, plus output tokens times the output price. The trap is the ratio. Output costs four to five times what input does on most providers, and reasoning models bill their thinking as output, so a switch to a reasoning model can triple a bill without a single extra user. If you are estimating rather than reading a real invoice, count roughly 750 tokens per page of text, and remember that every turn of a conversation resends the history.
The self-hosted side: hardware, power, ops
Three parts, and only one is the GPU.
Hardware, spread over the months you expect to keep it. Thirty-six months is the standard assumption. Buy the whole machine, not just the card: power supply, board, memory and disk are real money on a 48GB build.
Electricity, which is average draw times 730 hours times your rate. A 450W card in Cyprus at $0.28/kWh costs about $92 a month if it never sleeps, and far less if it idles between requests. Enter an honest average rather than the peak.
Ops, expressed as a percentage on top. This is the line people leave out and then regret: updates, monitoring, the runtime that breaks after a driver upgrade, and the hour someone spends on it every week. Twenty percent is a light estimate for a single box that someone already knows how to run. If nobody on the team has done it before, the real figure is higher, and it is the reason companies hire it out.
Breakeven and payback are two different questions
Breakeven holds your input/output mix fixed and scales the volume until the two monthly bills match. It answers "how big do we have to get". Payback ignores the write-off schedule and asks how many months of API spend the hardware price is worth, once you subtract the power and ops you will pay either way. It answers "when does the purchase clear". A workload can be past breakeven and still have an eighteen-month payback, which matters a great deal if the project might be cancelled in twelve.
What the cost model does not capture
Batching. A GPU serving one request at a time wastes most of itself; a batching engine reads the weights once for the whole batch, so total throughput on a busy server is several times the single-stream number. That makes self-hosting look better than this calculator suggests when you have concurrent load, and worse when traffic is spiky and the box sits idle.
Quality. If the open model needs two attempts where the API model needed one, you have quietly doubled your token volume. Test with your own evaluation set before you sign a purchase order.
Privacy. There is no column for "the data never left our network", and for a lot of businesses that is the only column that matters.
The third option: verifiably private inference
The comparison above assumes two choices, but there is a middle path: running open models inside confidential-computing hardware. Intel TDX and SGX enclaves, and the confidential mode on recent NVIDIA GPUs, encrypt memory so that the operator of the machine cannot read what is in it. The useful part is attestation. Before you send a single prompt, the hardware proves cryptographically which code it is running, so "we never see your data" stops being a line in a privacy policy and becomes something you can check.
Providers such as Tinfoil already sell open-model inference this way at close to ordinary API prices, so you keep per-token economics and lose the trust problem. It fits the case where the data is sensitive but a contractual and cryptographic guarantee is acceptable; when regulation says the data physically stays in your building, you are back to the hardware column of this calculator. We have built on SGX and TDX and can set up either.
Self-hosting economics, answered
At what volume does self-hosting an LLM become cheaper?
It depends on the model you are replacing, not on some universal number.
Replacing a frontier-class API at $5 in / $25 out, a single 24GB GPU tends to pay for itself within tens of millions of tokens a month. Replacing a cheap small model at $0.25 in / $1.50 out, the crossover sits far higher and often never arrives.
Put your own numbers in the calculator above.
What costs am I forgetting?
Electricity is the one people remember and the one that rarely decides it.
The costs that actually move the answer are the engineer who keeps the thing running, redundancy for when the box dies, and the quality gap if the open model needs more prompting or more retries than the API model did. The ops percentage in the calculator is where you price all of that.
Is an open model good enough to replace a commercial API?
For classification, extraction, summarisation, retrieval answers and most support flows, current open models in the 26-120B class (Gemma 4, Qwen3.6, gpt-oss) are good enough and the difference does not show up in your metrics. For the hardest reasoning and agentic work over long horizons, closed models still lead.
A lot of production systems end up mixed: an open model on the volume work, an API for the hard cases.
Does renting a cloud GPU change the maths?
Yes, and usually not in your favour for steady workloads. An hourly GPU rental turns capex into a much larger recurring cost, so the payback logic disappears and you are comparing two operating bills.
Rental makes sense for bursts, for evaluation before you buy, and when you cannot host hardware at all. Enter the monthly rental as the hardware cost with a one-month write-off period to model it.
What if our data cannot leave the building?
Then the calculation is a budget exercise, not a decision. Regulated data, client confidentiality and contractual data-residency terms rule out sending text to a third party at any price. That is the case Dubir's on-premise AI work is built for, and the number this tool gives you is what it will cost.
If the constraint is softer, along the lines of "nobody may be able to read our prompts", confidential-computing inference can satisfy it without buying hardware; see the section above.
If the numbers say build it, we build it
Dubir deploys private AI both ways: on hardware in your building, and as verifiably private inference in confidential-computing enclaves (Intel TDX/SGX) when buying a box is not the right move. Bring your numbers to a call and we will tell you honestly which side of the line your workload sits on.