NVIDIA H200 NVL (141 GiB)
NVIDIA · Dedicated memory
- Installed memory
- 141 GiB
- Usable per device
- 141 GiB
- Bandwidth
- 4,800 GB/s
- Largest fitting model
- 140.62B
- Power
- 600 W
Inference configuration
One sequence per batch, a 512-token prompt and 128 generated tokens. Multi-GPU estimates use pipeline parallelism over PCIe 4.
Open in calculatorDecode speed by model
Compare up to four checkpoints at the settings above. Bars show generated tokens per second.
Model performance matrix
Reference rates use the published 32K-cache benchmark when the calculator matches it. Other speeds use its roofline estimator; TTFT covers prompt processing.
| # | Quantization | Rate source | |||||
|---|---|---|---|---|---|---|---|
| 1 | SmolLM2-135M | 0.13B | 32,647.1 | 0.001 s | Fits | Q4 | Roofline |
| 2 | SmolLM2-135M-Instruct | 0.13B | 32,647.1 | 0.001 s | Fits | Q4 | Roofline |
| 3 | SmolLM2-360M | 0.36B | 12,737 | 0.001 s | Fits | Q4 | Roofline |
| 4 | SmolLM2-360M-Instruct | 0.36B | 12,737 | 0.001 s | Fits | Q4 | Roofline |
| 5 | Qwen2-0.5B | 0.49B | 10,086.4 | 0.002 s | Fits | Q4 | Roofline |
| 6 | Qwen2-0.5B-Instruct | 0.49B | 10,086.4 | 0.002 s | Fits | Q4 | Roofline |
| 7 | Qwen2.5-0.5B | 0.49B | 10,086.4 | 0.002 s | Fits | Q4 | Roofline |
| 8 | Qwen2.5-0.5B-Instruct | 0.49B | 10,086.4 | 0.002 s | Fits | Q4 | Roofline |
| 9 | Qwen2.5-Coder-0.5B | 0.49B | 10,086.4 | 0.002 s | Fits | Q4 | Roofline |
| 10 | Qwen2.5-Coder-0.5B-Instruct | 0.49B | 10,086.4 | 0.002 s | Fits | Q4 | Roofline |
Inference capacity
Llama 3.1 8B Instruct · Fits · 6.52 / 141 GiB
Selection notes
Largest catalog checkpoint inside this memory budget · zephyr-orpo-141b-A35b-v0.1 Use the remaining memory as headroom before increasing concurrency.
Fine-tuning capacity
Llama 3.1 8B Instruct · 4,096 tokens · batch 1 · calculator defaults
QLoRA
8.4 GiBFits
LoRA
20.8 GiBFits
Full fine-tuning
154.1 GiBOver memory
Frequently asked questions
How is usable memory determined?
The calculator uses the device profile’s available fraction · 100% × 141 GiB. System use can reduce the amount available at runtime.
Why is a speed missing?
Speed needs a fitting checkpoint and either matching calibration or both bandwidth and a comparable dense BF16 peak.