NVIDIA H800 SXM5 (80GB)

85.9 GB · 3,350 GB/s · CUDA · Dedicated memory

Installed memory
85.9 GB
Usable per device
85.9 GB
Bandwidth
3,350 GB/s
Largest fitting model
116.83B
Power
700 W

Inference configuration

One sequence per batch, a 512-token prompt and 128 generated tokens. Multi-GPU estimates use pipeline parallelism over PCIe 4.

Open in calculator

Model comparison

Compare up to four checkpoints at the settings above. The memory curve ends at each checkpoint’s context limit.

GB0.037.675.1112.72k4k8k16k32k64k128kLlama 3.1 8B Instruct · 2048 · 6.7 GBLlama 3.1 8B Instruct · 4096 · 7.0 GBLlama 3.1 8B Instruct · 8192 · 7.6 GBLlama 3.1 8B Instruct · 16384 · 8.8 GBLlama 3.1 8B Instruct · 32768 · 11.2 GBLlama 3.1 8B Instruct · 65536 · 16.0 GBLlama 3.1 8B Instruct · 131072 · 25.6 GBgpt-oss-120b · 2048 · 81.0 GBgpt-oss-120b · 4096 · 81.1 GBgpt-oss-120b · 8192 · 81.3 GBgpt-oss-120b · 16384 · 81.6 GBgpt-oss-120b · 32768 · 82.3 GBgpt-oss-120b · 65536 · 83.7 GBgpt-oss-120b · 131072 · 86.4 GBQwen2.5-72B · 2048 · 51.5 GBQwen2.5-72B · 4096 · 52.2 GBQwen2.5-72B · 8192 · 53.7 GBQwen2.5-72B · 16384 · 56.7 GBQwen2.5-72B · 32768 · 62.7 GBQwen2.5-72B · 65536 · 74.8 GBQwen2.5-72B · 131072 · 98.8 GBAvailable · 85.9 GB
Llama 3.1 8B Instruct7.0 GB
gpt-oss-120b81.1 GB
Qwen2.5-72B52.2 GB

Model performance matrix

Reference rates use the published 32K-cache benchmark when the calculator matches it. Other speeds use its roofline estimator; TTFT covers prompt processing.

#QuantizationRate source
1SmolLM2-135M0.13B22,7850 sFitsQ4Theory · BF16
2SmolLM2-135M-Instruct0.13B22,7850 sFitsQ4Theory · BF16
3SmolLM2-360M0.36B8,889.40.001 sFitsQ4Theory · BF16
4SmolLM2-360M-Instruct0.36B8,889.40.001 sFitsQ4Theory · BF16
5Qwen2-0.5B0.49B7,039.50.002 sFitsQ4Theory · BF16
6Qwen2-0.5B-Instruct0.49B7,039.50.002 sFitsQ4Theory · BF16
7Qwen2.5-0.5B0.49B7,039.50.002 sFitsQ4Theory · BF16
8Qwen2.5-0.5B-Instruct0.49B7,039.50.002 sFitsQ4Theory · BF16
9Qwen2.5-Coder-0.5B0.49B7,039.50.002 sFitsQ4Theory · BF16
10Qwen2.5-Coder-0.5B-Instruct0.49B7,039.50.002 sFitsQ4Theory · BF16

Inference capacity

Q4

116.83B

gpt-oss-120b

Q8

68.98B

Llama-2-70b-chat-hf

FP16

34.39B

Yi-34B-200K

Fine-tuning capacity

QLoRA 4-bit

72.71B

Qwen2.5-72B

LoRA 16-bit

33.34B

deepseek-coder-33b-base

Full fine-tuning

4.02B

Qwen3-4B

Selection notes

Best inference range

The fastest fitting checkpoints in this matrix span 0.13–0.36B.

At these settings, catalog checkpoints fit up to 116.83B · gpt-oss-120b

Bandwidth and throughput

3,350 GB/s · Llama 3.1 8B Instruct · 436.5 tok/s

Memory headroom

Required / available · 7.0 / 85.9 GB

Frequently asked questions

Which models run best?

SmolLM2-135M · 22,785 tok/s / SmolLM2-135M-Instruct · 22,785 tok/s / SmolLM2-360M · 8,889.4 tok/s

How accurate are these estimates?

Matched measurements are labeled Benchmark. Theoretical rates use memory bandwidth and the indicated compute precision. Backend, kernels and prompt length affect actual timings.

How is usable memory determined?

The calculator uses the device profile’s available fraction · 100% × 85.9 GB. System use can reduce the amount available at runtime.

Why is a speed missing?

Models beyond the memory or context limit have no speed estimate. For runnable models without a measured reference, the calculator uses a roofline estimate. Missing hardware inputs are labeled as scenario assumptions.