It's often the first objection we hear: bringing AI in-house seems to require an out-of-scale hardware investment. In practice, sizing depends on three variables — model size, quantization technique, real request volume — and almost never requires what's imagined at the outset.
The sizes that matter
- 7-8B models: run well on a single professional GPU or even workstation-grade hardware, ideal for internal assistants and RAG at modest volumes.
- 13-14B models: require a GPU with more dedicated memory, remain manageable on a single card with quantization.
- 70B models: typically require multiple GPUs in parallel, or aggressive quantization to fit on a smaller configuration.
- MoE models (mixture-of-experts, like some Mistral or DeepSeek variants): activate only part of the parameters per request, so the resource calculation differs from a dense model of the same nominal size.
Quantization changes the math
Reducing the numerical precision of the weights (from FP16 to INT8 or INT4) significantly cuts the memory required, and often increases response speed, at a quality cost that for many business use cases is imperceptible. It's often the single most effective lever for getting a model into production, on reasonable hardware, that would otherwise require much more expensive infrastructure.
Sovereign cloud or your own infrastructure?
This isn't an ideological choice but one of cash flow and predictability. Owning the hardware makes sense when usage volume is stable and predictable over time: the investment pays off and marginal costs fall. A Swiss sovereign cloud makes sense when volumes are variable, the project is still being validated, or you want to avoid the initial capital commitment while still keeping data on Swiss soil.
The calculation we do with clients
- How many requests per day, realistically, and with what peaks?
- What latency is acceptable for the end user?
- Does the data involved require physical isolation, or is Swiss residency enough?
- Do you prefer a capital investment (your own hardware) or a variable operating cost (cloud)?
Only after answering these questions does it make sense to talk about specific GPUs and configurations. Starting from the hardware before the use case is the most common way to spend twice what's needed — or the opposite, undersizing and finding out only in production.