Hardware

Apple M5 Ultra in the Mac Studio (available September 22, 2026): 30-core CPU, 64-core GPU with Neural Accelerators, 32-core Neural Engine and 1.2 TB/s unified…

M3 Ultra — up to 512GB unified in Mac Studio top spec. 819 GB/s bandwidth.

M4 Max — 546 GB/s memory bandwidth, up to 128GB unified. Succeeded by M5 Max (614 GB/s) in MacBook Pro.

M2 Ultra — up to 192GB at 800 GB/s. Mac Studio and Mac Pro hosting models.

M3 Max — 400 GB/s bandwidth, up to 128GB.

Original Ultra — 800 GB/s. 64–128GB unified. Still capable for 70B Q4.

Mid-tier M4 — 273 GB/s bandwidth, up to 64GB in Mac mini (48GB in MacBook Pro).

Copilot+ PC reference SoC. 12-core Oryon CPU + Adreno GPU + Hexagon NPU at 45 TOPS INT8. The first ARM Windows laptop with serious NPU compute; runs Phi Silica…

M2 Max — 400 GB/s bandwidth, up to 96GB.

Original M1 Max. 400 GB/s. 32–64GB unified.

Lower-tier Snapdragon X. 45 TOPS NPU.

Blackwell flagship. 32GB GDDR7 on a 512-bit bus delivers ~1.79 TB/s memory bandwidth — the new top of consumer hardware for local LLM inference. Holds 32B Q4…

Mobile Blackwell flagship. 24GB GDDR7 in a laptop is the new high-water mark.

Second-tier Blackwell. 16GB GDDR7, ~960 GB/s bandwidth. Fastest 16GB consumer card on the market.

Mobile Ada flagship. 16GB VRAM in a laptop. Premium gaming and AI laptop default.

Highest-tier Ampere consumer card. Used market gold for AI: 24 GB.

AMD's 24GB challenger to the 4090. ROCm Linux now solid for llama.cpp and vLLM.

24 GB GDDR6X at about 1 TB/s. Check 32B Q4 models against your context budget; 70B Q4 weights exceed this card's memory and require offload or multiple GPUs.

20GB RDNA 3. Cheaper alternative to XTX.

Refreshed 6900 XT with faster GDDR6 (576 GB/s). 16 GB VRAM, slightly more compute. AMD does not list this card as supported for ROCm, though Ollama lists it…

Ampere flagship-minus-one. 12 GB GDDR6X at 912 GB/s — closer to the 3090 in raw bandwidth than to the 3080. Fits 13B Q4 with full context, 32B Q4 with offload.…

The original 24GB CUDA value pick. Used market still strong in 2026 — many AI hobbyists run dual 3090 setups for 70B inference.

RDNA 2 enthusiast. 16 GB VRAM, 512 GB/s bandwidth, more compute units than the base 6800. AMD does not list this card as supported for ROCm, though Ollama…

RDNA 2 flagship at its 2020 launch; the RX 6950 XT followed in 2022. 16 GB VRAM at 512 GB/s, more compute than the 6800 XT. AMD does not list this card as…

Turing flagship. 11 GB GDDR6 at 616 GB/s — fits 13B Q4 comfortably. It competes with used 3060 12 GB on raw VRAM and beats it on compute.

16GB RDNA 4 at sub-$600. ROCm + Vulkan supported.

RDNA 4 flagship. 16GB at $599 — best AMD value for local AI in 2026.

16GB Blackwell at the upper-mid price tier. Fits up to 14B at Q5 fully on GPU; 32B Q4 needs partial offload.

16GB upgrade of the 4070 Ti. Solid mid-high pick for local AI.

Refreshed 4080 with 16GB GDDR6X. Slightly behind 5080 but well-supported.

16GB RDNA 3 mid-range.

RDNA 3 'Golden Rabbit Edition' — 16 GB at 576 GB/s, between the 6800 XT and 7900 XT. ROCm officially supported. U.S. launch price was $549. Comfortable for any…

12GB Ada — fits 7B–14B Q4 with usable context.

Original 4080. 16GB GDDR6X. Still capable for 14B Q4 work; 32B Q4 weights (about 19 GB) need partial offload.

Mid-life 12GB refresh of the 3080. Decent 7B–14B card on the used market.

Refreshed 6700 XT with faster GDDR6 (432 GB/s). 12 GB VRAM fits 13B Q4 comfortably. AMD does not list this card as supported for ROCm.

Laptop variant of Ampere. 16GB VRAM in a portable form factor was rare and remains a sleeper pick on the used market.

12 GB RDNA 2 at $479 MSRP. AMD does not list this card as supported for ROCm. The VRAM headroom matters — fits 13B Q4 comfortably.

Ampere Ti with GDDR6X at 608 GB/s. 8 GB VRAM is the ceiling — same as base 3070, no improvement on the variable that matters. The 'paid for the Ti suffix, got…

16 GB RDNA 2 — the AMD answer to the 4070 12 GB question. Comfortable for any 7B/13B model + 32B Q4 with offload. AMD does not list this card as supported for…

Original 10GB 3080. Tight on VRAM for AI but still capable for 7B work.

Ampere mid-high with 8 GB at 448 GB/s. Comfortable for 7B Q4 and 13B Q4 with light offload. The 'middle' of Ampere — better bandwidth than the 3060 12GB, less…

RDNA 1 flagship. AMD's ROCm support list does not include this card. Vulkan via llama.cpp is the only operator-grade path; performance is bandwidth-OK but…

Turing 'almost-flagship'. 8 GB VRAM is the ceiling — same as base 2080 — but more bandwidth (496 GB/s) and Tensor compute. The 8 GB ceiling matters: 13B fits…

The Turing refresh of the RTX 2070, rated at 73 Tensor TFLOPs by NVIDIA. Same 8 GB ceiling as the base 2070 but more compute. Strong used-market value in 2026.

Turing high-tier. 8 GB VRAM, similar bandwidth to the 2060 Super, slightly more compute. Runs 7B Q4, and 13B Q4 with light offload. A solid used-market choice.

Pascal halo card. 11 GB GDDR5X at 484 GB/s — outperforms many newer mid-range cards on raw bandwidth. 13B Q4 fits comfortably. The legendary 'still relevant'…

The 16GB sub-$500 sweet spot. Best value for entering local AI seriously.

AMD's RDNA 4 mainstream card. 16GB VRAM, ROCm + Vulkan support, $349 MSRP for the 16GB card ($299 for the 8GB card). NVIDIA's RTX 5060 Ti 16GB launched at…

Mid-range Blackwell with 12GB. 7B-14B Q4 territory.

10GB Battlemage at a $219 launch price. Entry budget compute.

8GB Blackwell. Capable of 7B-class models at Q4 or Q5 — go 16GB SKU instead for AI work.

Sub-$330 16GB AMD. Memory-bandwidth-limited but great VRAM-per-dollar.

Refreshed 4070. Strong mid-range value for 12GB-tier local AI.

Battlemage architecture. 12GB at a $249 launch price — the budget compute card. llama.cpp SYCL and Vulkan are usable paths for AI.

Chinese third-party modification of the stock RTX 2080 Ti, replacing the 11 GB GDDR6 with 22 GB. The TU102 chip, 352-bit memory bus, and 616 GB/s bandwidth are…

The poster child of 'cheap 16GB CUDA card'. Memory bandwidth is mediocre but 16GB opens up 14B Q4.

Original 4070. 12GB Ada. Now eclipsed by 4070 Super at the same price.

12GB RDNA 3.

8GB version — go 16GB SKU for AI work.

Alchemist 16GB. Vulkan llama.cpp is the most-tested route.

Refreshed 6600 XT with slightly faster GDDR6 (280 GB/s). 8 GB VRAM ceiling unchanged. Same buyer-decision shape as the 6600 XT — VRAM is the limiter, the…

The community pick for 'cheapest CUDA card with serious VRAM'. The value floor for local AI in 2026.

Entry RDNA 2. 8 GB VRAM, lower bandwidth (224 GB/s) — the bottleneck on AI. AMD does not list this card as supported for ROCm. Reasonable budget AMD entry but…

RDNA 2 mid-tier. 8 GB GDDR6 at 256 GB/s. AMD does not list this card as supported for ROCm. Bandwidth-bottlenecked vs the 6700 XT/6750 XT siblings. Reasonable…

8GB Ampere. Fits 7B Q4 only.

RDNA 1 mid-tier. 6 GB VRAM is the ceiling — fits 7B Q4 only with short context. No production ROCm support; Vulkan-only. The 'I have one in my 2020 build'…

Turing mid with the 8 GB upgrade — meaningful for AI. 7B Q4 fits comfortably with full context, 13B Q4 fits with offload. The '8 GB Turing' floor that many…

NVIDIA's entry point to the RTX 20 series, with Tensor cores. 6 GB VRAM is the bottleneck — 7B Q4 fits with limited context. NVIDIA rates its Tensor cores at…

Turing mid-tier without RT/Tensor cores. 6 GB VRAM fits 7B Q4 with short context. Bandwidth (288 GB/s) is solid for the tier. Same VRAM ceiling as the 1660…

Turing mid-range with GDDR6 — bandwidth jumps to 336 GB/s vs the base 1660. Same 6 GB VRAM ceiling. Still no Tensor cores. Strong used-market value in 2026.

Turing mid-range without RT cores. 6 GB VRAM fits 7B Q4 with short context. No Tensor cores on the GTX 16 series. Common in pre-built budget gaming rigs.

Pascal Ti slot between the 1070 and 1080. 8 GB GDDR5 at 256 GB/s. No FP16 acceleration on consumer Pascal — quantized inference only. The forgotten Ti — same…

AMD Polaris with 8 GB VRAM. Cheap on used market but AMD's ROCm README listed the RX 580 as officially supported in ROCm 3.0 and no longer did by ROCm 4.5, so…

Top GeForce GTX Pascal card until the GTX 1080 Ti arrived in March 2017. 8 GB GDDR5X at 320 GB/s — better bandwidth than the 1070. Fits 7B Q4; 13B Q4 with…

Pascal high-tier with 8 GB VRAM. Comfortable for 7B Q4 models. Bandwidth-limited like the rest of Pascal but the 8 GB headroom matters — fits 7B with…

CDNA 4. 288GB HBM3E.

Pro Blackwell — 96GB GDDR7 ECC. The single-card answer to 70B and 100B+ local inference.

NVIDIA RTX PRO 5000 is NVIDIA's Blackwell-generation workstation card, slotting between the consumer RTX 5090 (32GB) and the RTX PRO 6000 Blackwell (96GB).…

72-GPU Blackwell rack with 36 Grace CPUs. Hyperscale-only — relevant context here for understanding 'what frontier training runs on'.

256GB HBM3e — direct competitor to NVIDIA H200 with more memory.

Datacenter Blackwell. 180GB HBM3e per GPU on HGX and DGX B200 (186GB on GB200), ~8 TB/s bandwidth. Cloud-tier — you rent these by the hour.

The PCIe-form-factor variant of the H200. Same 141 GB HBM3e, same memory subsystem (~4.8 TB/s bandwidth), in a dual-slot PCIe card with a passive heat sink for…

Hopper refresh — 141GB HBM3e at ~4.8 TB/s. Datacenter-class; rentable on RunPod, Lambda, etc.

Intel's enterprise AI accelerator. 128GB HBM2e. Habana stack required — limited ecosystem support.

Third-party physical modification of a stock GIGABYTE / ASUS / MSI RTX 4090. Chinese specialty shops (and GPVLab in the US) rebuild the card to 48 GB. GPVLab…

192GB HBM3 datacenter card. Used by Microsoft, Oracle, Meta cloud deployments.

PCIe H100 card with 94GB HBM3 at 3.9 TB/s. Two bridged cards give 188GB combined. Built for LLM serving.

Ada-gen datacenter card. 48GB GDDR6 — popular at cloud GPU rentals as a budget H100 alternative.

32GB workstation Ada. Mid-tier pro card.

Inference-focused Ada datacenter card. Low-power 24GB suitable for 7B-14B serving.

Previous-gen Habana accelerator. 96GB HBM2e.

Hopper SXM5 — 80GB HBM3 at 3.35 TB/s. Cloud-rentable.

PCIe Hopper. Lower power, lower bandwidth than SXM. Server-tier.

64GB CDNA 2. Lower-power AMD datacenter option.

Pro Ada — 48GB ECC. Pre-Blackwell workstation default.

Original Ada datacenter. 48GB GDDR6.

CDNA 2. 128GB HBM2e. Powered the Frontier supercomputer.

24GB Ampere workstation card. Tighter power envelope than RTX 3090.

Ampere datacenter flagship. 80GB HBM2e at 2 TB/s. Still common at cloud providers.

Ampere workstation/datacenter hybrid. 48GB GDDR6.

Ampere-gen workstation card with 48GB. Common in AI labs; used market is reasonable for 48GB at this point.

Original A100. 40GB HBM2 at 1.55 TB/s.

Entry Blackwell. 8GB limits to 7B Q4 with limited context.

Intel Lunar Lake's Arc 140V iGPU (Xe2 Battlemage architecture). Bandwidth is about 137 GB/s (LPDDR5x-8533, by arithmetic). Pairs with Intel's NPU 4 (up to 48…

AMD's 880M iGPU (Ryzen AI 300 series Strix Point). RDNA 3.5 with unified memory up to LPDDR5x-8000, which is 128 GB/s peak by arithmetic, against 120 GB/s for…

Entry-level Ada. 8GB limits to 7B Q4.

AMD's 780M iGPU (Ryzen 7040 series Phoenix and Ryzen 8040 series Hawk Point). Shares system RAM via unified memory architecture; 32 GB DDR5 system gives…

Ampere entry with full Tensor + RT cores. 8 GB VRAM is the practical floor for serious 7B work. Bandwidth (224 GB/s) is the bottleneck. Often shipped in…

RDNA 1 entry. 8 GB GDDR6 at 224 GB/s. AMD does not list RDNA 1 cards as supported for ROCm — Vulkan via llama.cpp is the only operator-grade path. Buy this…

Turing entry refresh with GDDR6. 4 GB VRAM is below the practical AI floor — 1-3B Q4 only. No Tensor cores. Common in pre-built office PCs from 2019-2021 that…

Turing entry without RT/Tensor cores. 4 GB VRAM keeps it at the practical floor — 1-3B Q4 only. The 'I built a budget gaming PC' audience runs into VRAM walls…

Cut-down RX 580 with 4 GB VRAM. Below the practical AI floor; 1-3B Q4 with offload at best. AMD's ROCm 3.0 README listed the RX 570 as enabled without full…

Pascal mid-range, 6 GB VRAM. The most-installed Steam GPU for many years; high probability the 'I have a GTX 1060' audience is asking about this card. 7B Q4…

Pascal-era entry GPU. 4 GB VRAM is the practical floor for any local model — fits 1-3B at Q4 with room for short context. CUDA-compatible but no FP16…

Pascal mid-range cut down to 3 GB VRAM. Below the practical AI floor — even 3B Q4 models need ~2 GB plus KV cache. Operators with this card almost universally…

Mobile-only Ampere with 4 GB VRAM at 192 GB/s. The 4 GB ceiling is the bottleneck — 1-3B Q4 only with no headroom for context. CUDA + Tensor cores work, but…

Top-end Windows AI laptop with 24GB RTX 5090 Mobile.

Desktop-replacement gaming/AI laptop with cooler thermals than ultraslims.

16-inch M4 Max — 128GB unified at 546 GB/s. Apple's current 16-inch MacBook Pro uses M5 Pro and M5 Max.

Modular AMD laptop. Limited GPU but the platform is the appeal.

Top-spec Mac Studio with M3 Ultra. Up to 512GB unified memory in custom configs.

NVIDIA's desktop AI box — Grace Blackwell GB10 with 128GB unified LPDDR5X. The closest consumer can get to running 200B-class models locally without renting…

AMD's first true Strix Halo chip, combining Zen 5 CPU cores, RDNA 3.5 integrated graphics, and a 50-TOPS XDNA 2 NPU in one processor. The headline feature for…

Combined CPU + GPU APU with 128GB unified HBM3. Powers the El Capitan supercomputer.

iPad Pro M4 — 9- or 10-core CPU + 10-core GPU + 16-core Neural Engine (38 TOPS). 8/16GB unified memory. Apple's iPad Pro has since moved to M5.

Late-2024 Android flagship SoC. Oryon CPU + Hexagon NPU. 8B-class models become viable on-device with adequate quantization.

iPhone 16 Pro SoC. Improved Neural Engine for Apple Intelligence on-device workloads. 8GB RAM as the new mobile floor enables 3B-class on-device models.

Pixel 9 SoC. Google's mobile chip optimized for Gemini Nano + on-device transcription / summarization. NPU TOPS not publicly disclosed by Google.

Flagship Android SoC with a Hexagon NPU. Qualcomm says its AI Engine runs generative AI models up to 10B parameters on-device.

iPhone 15 Pro / 15 Pro Max SoC. 6-core CPU + 6-core GPU + 16-core Neural Engine. First A-series with hardware ray tracing; runs MLX-Swift and CoreML for…

Strix Point laptop SoC. XDNA 2 NPU at 50 TOPS INT8 + RDNA 3.5 iGPU. ROCm support on Linux unlocks llama.cpp ROCm path; on Windows, ONNX Runtime + DirectML.

Intel Lunar Lake 8-core. NPU 4 at 47 TOPS INT8 + Xe2 iGPU + Skymont E-cores. Copilot+ PC certified. Runs DirectML + ONNX Runtime + OpenVINO; primary…