Hardware directory

Find hardware for your model and compare its memory budget and inference speed.

162 matches / 162 total

Capacity includes the calculator’s safety margin. Largest checkpoint at Q4 · 4K · KV FP16 · batch 1

CompareSave
A100 80 GiBDedicated80 / 80 GiB2,039 GB/s400 W116.83B
AMD RX 6600 (8 GiB)Dedicated8 / 8 GiB224 GB/s132 W8.83B
AMD RX 6600 XT (8 GiB)Dedicated8 / 8 GiB256 GB/s160 W8.83B
AMD RX 6650 XT (8 GiB)Dedicated8 / 8 GiB280 GB/s180 W8.83B
AMD RX 6700 (10 GiB)Dedicated10 / 10 GiB320 GB/s175 W10.73B
AMD RX 6700 XT (12 GiB)Dedicated12 / 12 GiB384 GB/s230 W15.71B
AMD RX 6750 XT (12 GiB)Dedicated12 / 12 GiB432 GB/s250 W15.71B
AMD RX 6800 (16 GiB)Dedicated16 / 16 GiB512 GB/s250 W20.91B
AMD RX 6800 XT (16 GiB)Dedicated16 / 16 GiB512 GB/s300 W20.91B
AMD RX 6900 XT (16 GiB)Dedicated16 / 16 GiB512 GB/s300 W20.91B
AMD RX 6950 XT (16 GiB)Dedicated16 / 16 GiB576 GB/s335 W20.91B
AMD RX 7600 (8 GiB)Dedicated8 / 8 GiB288 GB/s165 W8.83B
AMD RX 7600 XT (16 GiB)Dedicated16 / 16 GiB288 GB/s190 W20.91B
AMD RX 7700 XT (12 GiB)Dedicated12 / 12 GiB432 GB/s245 W15.71B
AMD RX 7800 XT (16 GiB)Dedicated16 / 16 GiB624 GB/s263 W20.91B
AMD RX 7900 GRE (16 GiB)Dedicated16 / 16 GiB576 GB/s260 W20.91B
AMD RX 7900 XT (20 GiB)Dedicated20 / 20 GiB800 GB/s315 W27.78B
AMD RX 7900 XTX (24 GiB)Dedicated24 / 24 GiB960 GB/s355 W34.39B
AMD RX 9060 XT (16 GiB)Dedicated16 / 16 GiB320 GB/s180 W20.91B
AMD RX 9070 (16 GiB)Dedicated16 / 16 GiB640 GB/s220 W20.91B
AMD RX 9070 XT (16 GiB)Dedicated16 / 16 GiB640 GB/s304 W20.91B
AMD Ryzen AI 9 HX 370 (32 GiB system, 24 GiB GPU-accessible)Unified32 / 24 GiB89.6 GB/s54 W34.39B
AMD Ryzen AI 9 HX 370 (64 GiB system, 48 GiB GPU-accessible)Unified64 / 48 GiB102.4 GB/s54 W70.55B
AMD Ryzen AI 9 HX 370 (96 GiB system, 72 GiB GPU-accessible)Unified96 / 72 GiB128 GB/s54 W72.71B
AMD Ryzen AI MAX+ 395 (128 GiB system, 96 GiB GPU-accessible)Unified128 / 96 GiB256 GB/s120 W140.62B
Apple M1 Max 32 GiBUnified32 / 24 GiB400 GB/s—34.39B
Apple M1 Max 64 GiBUnified64 / 48 GiB400 GB/s—70.55B
Apple M2 Max (32 GiB)Unified32 / 24 GiB400 GB/s100 W34.39B
Apple M2 Max (64 GiB)Unified64 / 48 GiB400 GB/s100 W70.55B
Apple M2 Max (96 GiB)Unified96 / 72 GiB400 GB/s100 W72.71B
Apple M2 Pro (16 GiB)Unified16 / 12 GiB200 GB/s67 W15.71B
Apple M2 Ultra (64 GiB)Unified64 / 48 GiB800 GB/s230 W70.55B
Apple M2 Ultra (128 GiB)Unified128 / 96 GiB800 GB/s230 W140.62B
Apple M2 Ultra (192 GiB)Unified192 / 144 GiB800 GB/s230 W140.62B
Apple M3 Max (36 GiB)Unified36 / 27 GiB300 GB/s115 W34.39B
Apple M3 Max (48 GiB)Unified48 / 36 GiB400 GB/s115 W46.7B
Apple M3 Max (64 GiB)Unified64 / 48 GiB400 GB/s115 W70.55B
Apple M3 Max (96 GiB)Unified96 / 72 GiB400 GB/s115 W72.71B
Apple M3 Max (128 GiB)Unified128 / 96 GiB400 GB/s115 W140.62B
Apple M3 Pro (18 GiB)Unified18 / 13.5 GiB150 GB/s75 W15.71B
Apple M3 Pro (36 GiB)Unified36 / 27 GiB150 GB/s75 W34.39B
Apple M3 Ultra (256 GiB)Unified256 / 192 GiB819.2 GB/s250 W235.74B
Apple M3 Ultra (512 GiB)Unified512 / 384 GiB819.2 GB/s250 W480.15B
Apple M4 (16 GiB)Unified16 / 12 GiB120 GB/s45 W15.71B
Apple M4 (24 GiB)Unified24 / 18 GiB120 GB/s45 W23.57B
Apple M4 (32 GiB)Unified32 / 24 GiB120 GB/s45 W34.39B
Apple M4 Max (64 GiB)Unified64 / 48 GiB546 GB/s140 W70.55B
Apple M4 Max (96 GiB)Unified96 / 72 GiB546 GB/s140 W72.71B
Apple M4 Max (128 GiB)Unified128 / 96 GiB546 GB/s140 W140.62B
Apple M4 Pro (32 GiB)Unified32 / 24 GiB273 GB/s85 W34.39B
Apple M4 Pro (48 GiB)Unified48 / 36 GiB273 GB/s85 W46.7B
Apple M4 Pro (64 GiB)Unified64 / 48 GiB273 GB/s85 W70.55B
Apple M5 (16 GiB)Unified16 / 12 GiB153 GB/s50 W15.71B
Apple M5 (24 GiB)Unified24 / 18 GiB153 GB/s50 W23.57B
Apple M5 (32 GiB)Unified32 / 24 GiB153 GB/s50 W34.39B
Apple M5 Max (48 GiB)Unified48 / 36 GiB614 GB/s150 W46.7B
Apple M5 Max (64 GiB)Unified64 / 48 GiB614 GB/s150 W70.55B
Apple M5 Max (96 GiB)Unified96 / 72 GiB614 GB/s150 W72.71B
Apple M5 Max 128 GiBUnified128 / 96 GiB614 GB/s150 W140.62B
Apple M5 Pro (24 GiB)Unified24 / 18 GiB307 GB/s95 W23.57B
Apple M5 Pro (48 GiB)Unified48 / 36 GiB307 GB/s95 W46.7B
Apple M5 Pro (64 GiB)Unified64 / 48 GiB307 GB/s95 W70.55B
GB200 Blackwell GPU 186 GiBDedicated186 / 186 GiB8,000 GB/s1200 W235.74B
GTX 1050 Ti 4 GiBDedicated4 / 4 GiB112.1 GB/s75 W3.84B
GTX 1060 3 GiBDedicated3 / 3 GiB192 GB/s120 W3.09B
GTX 1060 6 GiBDedicated6 / 6 GiB192 GB/s120 W7.62B
GTX 1070 8 GiBDedicated8 / 8 GiB256 GB/s150 W8.83B
GTX 1070 Ti 8 GiBDedicated8 / 8 GiB256 GB/s180 W8.83B
GTX 1080 8 GiBDedicated8 / 8 GiB320 GB/s180 W8.83B
GTX 1080 Ti 11 GiBDedicated11 / 11 GiB484 GB/s250 W15.71B
GTX 1630 4 GiBDedicated4 / 4 GiB96 GB/s75 W3.84B
GTX 1650 GDDR5 4 GiBDedicated4 / 4 GiB128 GB/s75 W3.84B
GTX 1650 GDDR6 4 GiBDedicated4 / 4 GiB192 GB/s75 W3.84B
GTX 1650 SUPER 4 GiBDedicated4 / 4 GiB192 GB/s100 W3.84B
GTX 1660 6 GiBDedicated6 / 6 GiB192 GB/s120 W7.62B
GTX 1660 SUPER 6 GiBDedicated6 / 6 GiB336 GB/s125 W7.62B
GTX 1660 Ti 6 GiBDedicated6 / 6 GiB288 GB/s120 W7.62B
H100 80 GiBDedicated80 / 80 GiB3,350 GB/s700 W116.83B
H200 SXM 141 GiBDedicated141 / 141 GiB4,800 GB/s700 W140.62B
Instinct MI210 (64 GiB)Dedicated64 / 64 GiB1,638 GB/s300 W72.71B
Instinct MI250 128 GiBDedicated128 / 128 GiB3,200 GB/s500 W140.62B
L40S 48 GiBDedicated48 / 48 GiB864 GB/s350 W70.55B
Mac 16 GiB unified memoryUnified16 / 12 GiB——15.71B
Mac 24 GiB unified memoryUnified24 / 18 GiB——23.57B
Mac 36 GiB unified memoryUnified36 / 27 GiB——34.39B
Mac 48 GiB unified memoryUnified48 / 36 GiB——46.7B
Mac 64 GiB unified memoryUnified64 / 48 GiB——70.55B
Mac 128 GiB unified memoryUnified128 / 96 GiB——140.62B
Mac 192 GiB unified memoryUnified192 / 144 GiB——140.62B
NVIDIA A2 (16 GiB)Dedicated16 / 16 GiB200 GB/s60 W20.91B
NVIDIA A10G (24 GiB)Dedicated24 / 24 GiB600 GB/s300 W34.39B
NVIDIA A16 GPU 16 GiBDedicated16 / 16 GiB200 GB/s—20.91B
NVIDIA A30 24 GiBDedicated24 / 24 GiB933 GB/s165 W34.39B
NVIDIA A40 (48 GiB)Dedicated48 / 48 GiB696 GB/s300 W70.55B
NVIDIA A100 (40 GiB)Dedicated40 / 40 GiB1,555 GB/s400 W57.41B
NVIDIA A800 (40 GiB)Dedicated40 / 40 GiB1,555.2 GB/s400 W57.41B
NVIDIA A800 (80 GiB)Dedicated80 / 80 GiB1,935 GB/s400 W116.83B
NVIDIA B100 (192 GiB)Dedicated192 / 192 GiB8,000 GB/s1000 W235.74B
NVIDIA B200 192 GiB (ApX)Dedicated192 / 192 GiB8,000 GB/s1200 W235.74B
NVIDIA Blackwell B300 (288 GiB)Dedicated288 / 288 GiB8,000 GB/s1000 W405.85B
NVIDIA GB10 Grace Blackwell (128 GiB) - DGX SparkUnified128 / 128 GiB273.2 GB/s140 W140.62B
NVIDIA H20 (96 GiB)Dedicated96 / 96 GiB4,000 GB/s400 W140.62B
NVIDIA H100 NVL (94 GiB)Dedicated94 / 94 GiB3,938 GB/s700 W140.62B
NVIDIA H100 PCIe (80 GiB)Dedicated80 / 80 GiB2,000 GB/s350 W116.83B
NVIDIA H200 NVL (141 GiB)Dedicated141 / 141 GiB4,800 GB/s600 W140.62B
NVIDIA H800 (80 GiB)Dedicated80 / 80 GiB2,000 GB/s700 W116.83B
NVIDIA HGX B200 180 GiBDedicated180 / 180 GiB7,700 GB/s1000 W235.74B
NVIDIA L4 24 GiBDedicated24 / 24 GiB300 GB/s72 W34.39B
NVIDIA L20 (48 GiB)Dedicated48 / 48 GiB864 GB/s275 W70.55B
NVIDIA L40 (48 GiB)Dedicated48 / 48 GiB864 GB/s300 W70.55B
NVIDIA RTX 2000 Ada Generation (16 GiB)Dedicated16 / 16 GiB224 GB/s70 W20.91B
NVIDIA RTX 3070 Ti (8 GiB)Dedicated8 / 8 GiB608 GB/s290 W8.83B
NVIDIA RTX 3080 (10 GiB)Dedicated10 / 10 GiB760.3 GB/s320 W10.73B
NVIDIA RTX 3080 (12 GiB)Dedicated12 / 12 GiB912 GB/s350 W15.71B
NVIDIA RTX 3080 Ti (12 GiB)Dedicated12 / 12 GiB912.4 GB/s350 W15.71B
NVIDIA RTX 3090 Ti (24 GiB)Dedicated24 / 24 GiB1,008 GB/s450 W34.39B
NVIDIA RTX 4070 SUPER (12 GiB)Dedicated12 / 12 GiB504.2 GB/s220 W15.71B
NVIDIA RTX 4070 Ti (12 GiB)Dedicated12 / 12 GiB504.2 GB/s285 W15.71B
NVIDIA RTX 4080 (16 GiB)Dedicated16 / 16 GiB716.8 GB/s320 W20.91B
NVIDIA RTX 4080 SUPER (16 GiB)Dedicated16 / 16 GiB736.3 GB/s320 W20.91B
NVIDIA RTX 4500 Ada Generation (24 GiB)Dedicated24 / 24 GiB432 GB/s210 W34.39B
NVIDIA RTX 4500 Blackwell (32 GiB)Dedicated32 / 32 GiB896 GB/s175 W46.7B
NVIDIA RTX 5000 Ada Generation (32 GiB)Dedicated32 / 32 GiB576 GB/s250 W46.7B
NVIDIA RTX 5000 Blackwell (48 GiB)Dedicated48 / 48 GiB1,344 GB/s250 W70.55B
NVIDIA RTX 5060 (8 GiB)Dedicated8 / 8 GiB448 GB/s130 W8.83B
NVIDIA RTX 5060 Ti (8 GiB)Dedicated8 / 8 GiB448 GB/s180 W8.83B
NVIDIA RTX 5060 Ti (16 GiB)Dedicated16 / 16 GiB448 GB/s185 W20.91B
NVIDIA RTX 5070 (12 GiB)Dedicated12 / 12 GiB672 GB/s220 W15.71B
NVIDIA RTX 5070 Ti (16 GiB)Dedicated16 / 16 GiB896 GB/s300 W20.91B
NVIDIA RTX 5080 (16 GiB)Dedicated16 / 16 GiB960 GB/s360 W20.91B
NVIDIA RTX 6000 Ada Generation (48 GiB)Dedicated48 / 48 GiB960 GB/s300 W70.55B
NVIDIA RTX 6000 Blackwell (96 GiB)Dedicated96 / 96 GiB1,792 GB/s500 W140.62B
NVIDIA RTX A4000 (16 GiB)Dedicated16 / 16 GiB448 GB/s140 W20.91B
NVIDIA RTX A5000 (24 GiB)Dedicated24 / 24 GiB768 GB/s230 W34.39B
NVIDIA RTX PRO 4000 Blackwell (24 GiB)Dedicated24 / 24 GiB672 GB/s145 W34.39B
NVIDIA RTX PRO 4000 Blackwell SFF (24 GiB)Dedicated24 / 24 GiB432 GB/s70 W34.39B
NVIDIA RTX PRO 5000 Blackwell (72 GiB)Dedicated72 / 72 GiB1,344 GB/s300 W72.71B
NVIDIA Rubin R100 (288 GiB)Dedicated288 / 288 GiB22,000 GB/s1000 W405.85B
NVIDIA T4 16 GiBDedicated16 / 16 GiB320 GB/s70 W20.91B
Radeon AI PRO R9700 (32 GiB)Dedicated32 / 32 GiB643.2 GB/s300 W46.7B
Radeon PRO W6600 (8 GiB)Dedicated8 / 8 GiB224 GB/s100 W8.83B
Radeon PRO W6800 (32 GiB)Dedicated32 / 32 GiB512 GB/s250 W46.7B
Radeon PRO W7500 (8 GiB)Dedicated8 / 8 GiB176 GB/s70 W8.83B
Radeon PRO W7600 (8 GiB)Dedicated8 / 8 GiB288 GB/s130 W8.83B
Radeon PRO W7700 (16 GiB)Dedicated16 / 16 GiB576 GB/s190 W20.91B
Radeon PRO W7800 (32 GiB)Dedicated32 / 32 GiB576 GB/s260 W46.7B
Radeon PRO W7900 (48 GiB)Dedicated48 / 48 GiB864 GB/s295 W70.55B
Radeon RX 580 8 GiBDedicated8 / 8 GiB256 GB/s185 W8.83B
RTX 3060 12 GiBDedicated12 / 12 GiB360 GB/s170 W15.71B
RTX 3060 Ti 8 GiBDedicated8 / 8 GiB448 GB/s200 W8.83B
RTX 3070 8 GiBDedicated8 / 8 GiB448 GB/s220 W8.83B
RTX 3090 24 GiBDedicated24 / 24 GiB936.2 GB/s350 W34.39B
RTX 4000 Ada Generation 20 GiBDedicated20 / 20 GiB360 GB/s130 W27.78B
RTX 4060 8 GiBDedicated8 / 8 GiB272 GB/s115 W8.83B
RTX 4060 Ti 8 GiBDedicated8 / 8 GiB288 GB/s160 W8.83B
RTX 4060 Ti 16 GiBDedicated16 / 16 GiB288 GB/s165 W20.91B
RTX 4070 12 GiBDedicated12 / 12 GiB504 GB/s200 W15.71B
RTX 4070 Ti Super 16 GiBDedicated16 / 16 GiB672.3 GB/s285 W20.91B
RTX 4090 24 GiBDedicated24 / 24 GiB1,008 GB/s450 W34.39B
RTX 4090 D 24 GiBDedicated24 / 24 GiB1,008 GB/s425 W34.39B
RTX 5090 32 GiBDedicated32 / 32 GiB1,792 GB/s575 W46.7B
RTX A6000 48 GiBDedicated48 / 48 GiB768 GB/s300 W70.55B

Filters

Architecture
Open hardware comparison →