Public APIs are unbeatable at low volume. At high volume, the per-token markup adds up. Private hosting flips the curve: you pay for capacity, not usage.
The crossover point
For a typical 70B parameter model, the public API might cost $0.40 per million input tokens and $1.00 per million output tokens. A dedicated GPU instance can process tens of millions of tokens per day for a fixed monthly cost.
For most SMEs, the crossover happens around 2-5 billion tokens per month, depending on model size and peak concurrency.
When private hosting wins
Beyond cost, private hosting gives you predictable latency, data residency, and the ability to run custom weights. For regulated industries, it is often the only viable option.