NVIDIA H800 SXM5 (80GB)
85.9 GB · 3,350 GB/s · CUDA · Dedicated memory
- Installed memory
- 85.9 GB
- Usable per device
- 85.9 GB
- Bandwidth
- 3,350 GB/s
- Largest fitting model
- 116.83B
- Power
- 700 W
Inference configuration
One sequence per batch, a 512-token prompt and 128 generated tokens. Multi-GPU estimates use pipeline parallelism over PCIe 4.
Open in calculatorModel comparison
Compare up to four checkpoints at the settings above. The memory curve ends at each checkpoint’s context limit.
Model performance matrix
Reference rates use the published 32K-cache benchmark when the calculator matches it. Other speeds use its roofline estimator; TTFT covers prompt processing.
| # | Quantization | Rate source | |||||
|---|---|---|---|---|---|---|---|
| 1 | SmolLM2-135M | 0.13B | 22,785 | 0 s | Fits | Q4 | Theory · BF16 |
| 2 | SmolLM2-135M-Instruct | 0.13B | 22,785 | 0 s | Fits | Q4 | Theory · BF16 |
| 3 | SmolLM2-360M | 0.36B | 8,889.4 | 0.001 s | Fits | Q4 | Theory · BF16 |
| 4 | SmolLM2-360M-Instruct | 0.36B | 8,889.4 | 0.001 s | Fits | Q4 | Theory · BF16 |
| 5 | Qwen2-0.5B | 0.49B | 7,039.5 | 0.002 s | Fits | Q4 | Theory · BF16 |
| 6 | Qwen2-0.5B-Instruct | 0.49B | 7,039.5 | 0.002 s | Fits | Q4 | Theory · BF16 |
| 7 | Qwen2.5-0.5B | 0.49B | 7,039.5 | 0.002 s | Fits | Q4 | Theory · BF16 |
| 8 | Qwen2.5-0.5B-Instruct | 0.49B | 7,039.5 | 0.002 s | Fits | Q4 | Theory · BF16 |
| 9 | Qwen2.5-Coder-0.5B | 0.49B | 7,039.5 | 0.002 s | Fits | Q4 | Theory · BF16 |
| 10 | Qwen2.5-Coder-0.5B-Instruct | 0.49B | 7,039.5 | 0.002 s | Fits | Q4 | Theory · BF16 |
Inference capacity
Q4
116.83Bgpt-oss-120b
Q8
68.98BLlama-2-70b-chat-hf
FP16
34.39BYi-34B-200K
Fine-tuning capacity
QLoRA 4-bit
72.71BQwen2.5-72B
LoRA 16-bit
33.34Bdeepseek-coder-33b-base
Full fine-tuning
4.02BQwen3-4B
Selection notes
Best inference range
The fastest fitting checkpoints in this matrix span 0.13–0.36B.
At these settings, catalog checkpoints fit up to 116.83B · gpt-oss-120b
Bandwidth and throughput
3,350 GB/s · Llama 3.1 8B Instruct · 436.5 tok/s
Memory headroom
Required / available · 7.0 / 85.9 GB
Frequently asked questions
Which models run best?
SmolLM2-135M · 22,785 tok/s / SmolLM2-135M-Instruct · 22,785 tok/s / SmolLM2-360M · 8,889.4 tok/s
How accurate are these estimates?
Matched measurements are labeled Benchmark. Theoretical rates use memory bandwidth and the indicated compute precision. Backend, kernels and prompt length affect actual timings.
How is usable memory determined?
The calculator uses the device profile’s available fraction · 100% × 85.9 GB. System use can reduce the amount available at runtime.
Why is a speed missing?
Models beyond the memory or context limit have no speed estimate. For runnable models without a measured reference, the calculator uses a roofline estimate. Missing hardware inputs are labeled as scenario assumptions.