Open Weights Changed the Buy-Versus-Build Calculation

Two years ago the practical answer to "should we run our own model" was almost always no. The capability gap was wide, the operational burden was real, and the hardware was scarce. The gap has narrowed considerably for a large class of tasks, and the question is now a genuine engineering trade-off rather than a foregone conclusion.

Three conditions that favour self-hosting

• Data that cannot leave. Regulatory or contractual constraints make this decision for you, and no amount of vendor assurance changes a legal requirement.
• High, predictable volume. Per-token pricing is excellent for spiky, low-volume workloads and punishing for steady high ones. There is a crossover point and it is closer than most teams assume.
• Narrow, repetitive tasks. Classification, extraction, and routing on a stable domain are exactly where a smaller tuned model matches a frontier one at a fraction of the cost.

Three conditions that favour an API

• Frontier reasoning requirements. If your product depends on the hardest capability tier, you are not going to reproduce it in house.
• Variable, unpredictable load. Paying for idle accelerators to survive a weekly peak is a bad trade.
• A small team. Serving infrastructure, capacity planning, and upgrade cycles are a real headcount cost that does not appear on the pricing page.

The hybrid that most teams land on

In practice the answer is usually both, split by task rather than by principle. Self-host the mechanical majority of your traffic where cost and latency dominate and quality requirements are modest. Call an API for the difficult tail. Put a routing layer between them so the split is a configuration change rather than an architectural commitment.

This also gives you leverage. A team that can move workloads between providers negotiates differently from one that cannot, and it survives a provider outage as a degradation rather than an incident.

Costs people forget

The honest total for self-hosting includes accelerator time at your actual utilisation, not peak; engineering time for serving, batching, and monitoring; the evaluation work to verify each model upgrade; and the opportunity cost of the features not built while doing all of it. Compare that against API spend measured over a realistic year rather than a demo month.

Model weights being open does not make inference free. It makes it yours, which is a different and sometimes better thing.

How to decide without a six-week study

Take one week of production traffic. Classify it by task. Estimate cost under each option using real token counts. Then ask whether any of that traffic carries data you would be uncomfortable seeing in a vendor incident report. Those two numbers and that one question will resolve the debate faster than any benchmark.