How much memory does your LLM need?
Estimate memory for model weights, context and concurrent requests before choosing a GPU.
Estimate the model weights
Start with parameters × bits per parameter ÷ 8. An 8-billion-parameter model is roughly 16 GB at 16-bit or 4 GB at 4-bit. These are decimal gigabytes and weight-only estimates, excluding quantization metadata and runtime memory.
Include memory for context
Generation uses a KV cache. Its size depends on the architecture, context length and concurrent requests. The weights may fit while a long conversation or several simultaneous requests exhaust available memory.
Leave headroom and measure the peak
Include engine memory and working buffers. There is no universal headroom percentage: test your maximum expected context and load. These estimates cover inference; training needs a separate calculation.
Choose a GPU for your workload
Compare specifications and rates in the catalog. Prepare your model, environment and preferred dates to discuss the configuration and access terms.
Keep exploring
Choosing a GPU for LLMs: RTX 4090, A100, H100 or GX10
GPU rental cost: how to estimate your budget