LLM GPU requirements

The formula and what it assumes

VRAM and minimum setup by model and precision

Smallest setup: fewest GPUs, then least combined VRAM, among GPUs with an offer right now. Cheapest setup: lowest hourly cost for the whole configuration, which is often more, smaller cards. Prices are the cheapest current on-demand price per GPU multiplied by the GPU count; a provider may not sell that exact count in one machine, so check the rent page. Prices observed 22 Sep 2026, 03:19 UTC.

What fits in 24, 48, 80, 141 and 192 GB

One GPU, one request, 8,192 tokens of context. Each size lists what it adds over the size before it, so a bigger card also runs everything above its row.

Full specs for each of these GPUs are in the GPU specs chart. FP8 rows run at full speed only on GPUs with FP8 hardware; the chart shows which publish an FP8 figure.

Model facts and where they come from

• Llama 4 Scout (17B active, 16 experts): 36 of 48 layers use chunked attention over 8,192 tokens. The parameter total includes the vision encoder.
• Llama 4 Maverick (17B active, 128 experts): 36 of 48 layers use chunked attention over 8,192 tokens. The parameter total includes the vision encoder.
• DeepSeek V3 (671B): Multi-head latent attention caches 576 values per token per layer (kv_lora_rank 512 plus qk_rope_head_dim 64). The repository also holds a 14B multi-token-prediction module that is not counted here.
• DeepSeek R1 (671B): Same architecture as DeepSeek V3: multi-head latent attention caching 576 values per token per layer.
• Gemma 3 12B: Five of every six layers use a 1,024-token sliding window. The parameter total includes the vision encoder.
• Gemma 3 27B: Five of every six layers use a 1,024-token sliding window. The parameter total includes the vision encoder.
• gpt-oss 20B: Published with MXFP4 expert weights; the checkpoint size is the sum of the repository's safetensors files. Half the layers use a 128-token sliding window.
• gpt-oss 120B: Published with MXFP4 expert weights; the checkpoint size is the sum of the repository's safetensors files. Half the layers use a 128-token sliding window.

Method and sources

Questions

Related

• Data Egress Costs by Cloud ProviderFree egress allowances and per-TB overage rates for GPU clouds, hyperscalers, budget clouds, object storage and CDNs, with ways to cut the bill.
• GPU Specs Comparison ChartEvery tracked GPU family side by side: VRAM, memory bandwidth, dense and with-sparsity throughput, TDP and the cheapest live rental price, checked against vendor datasheets.
• LLM VRAM CalculatorWork out how much VRAM a model needs at a given precision, context length and batch, and which GPU setups fit it at today's prices.
• Rent vs Buy GPU CalculatorCompare owning a GPU (purchase, power, colocation, resale) with renting it at the live cheapest and median price, and find the break-even month.