GTX 1070 8 GiB

NVIDIA · Dedicated memory

Installed memory
8 GiB
Usable per device
8 GiB
Bandwidth
256 GB/s
Largest fitting model
8.83B
Power
150 W

Inference configuration

One sequence per batch, a 512-token prompt and 128 generated tokens. Multi-GPU estimates use pipeline parallelism over PCIe 4.

Open in calculator

Decode speed by model

Compare up to four checkpoints at the settings above. Bars show generated tokens per second.

Llama 3.1 8B InstructNo data
DeepHermes-3-Llama-3-3B-PreviewNo data
DeepHermes-3-Llama-3-8B-PreviewNo data

Model performance matrix

Reference rates use the published 32K-cache benchmark when the calculator matches it. Other speeds use its roofline estimator; TTFT covers prompt processing.

#QuantizationRate source
1DeepHermes-3-Llama-3-3B-Preview3.21BNo dataNo dataFitsQ4—
2DeepHermes-3-Llama-3-8B-Preview8.03BNo dataNo dataFitsQ4—
3DeepHermes-3-Mistral-24B-Preview23.57BNo dataNo dataOver memoryQ4—
4DeepHermes-Financial-Fundamentals-Prediction-Specialist-Atropos8.03BNo dataNo dataFitsQ4—
5deepseek-coder-1.3b-instruct1.35BNo dataNo dataFitsQ4—
6deepseek-coder-33b-base33.34BNo dataNo dataOver memoryQ4—
7deepseek-coder-33b-instruct33.34BNo dataNo dataOver memoryQ4—
8deepseek-coder-6.7b-base6.74BNo dataNo dataFitsQ4—
9deepseek-coder-6.7b-instruct6.74BNo dataNo dataFitsQ4—
10deepseek-coder-7b-base-v1.56.91BNo dataNo dataFitsQ4—
1 / 33

Inference capacity

Llama 3.1 8B Instruct · Fits · 6.52 / 8 GiB

Selection notes

Largest catalog checkpoint inside this memory budget · Yi-1.5-9B Use the remaining memory as headroom before increasing concurrency.

Fine-tuning capacity

Llama 3.1 8B Instruct · 4,096 tokens · batch 1 · calculator defaults

QLoRA

8.4 GiB

Over memory

LoRA

20.8 GiB

Unsupported setup

Full fine-tuning

154.1 GiB

Unsupported setup

Frequently asked questions

How is usable memory determined?

The calculator uses the device profile’s available fraction · 100% × 8 GiB. System use can reduce the amount available at runtime.

Why is a speed missing?

Speed needs a fitting checkpoint and either matching calibration or both bandwidth and a comparable dense BF16 peak.