HBM (High Bandwidth Memory)
Definition pending
We've cataloged "HBM (High Bandwidth Memory)" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
HBM (High Bandwidth Memory) is the ultra-fast memory used in datacenter GPUs (A100, H100). HBM3 on H100 delivers 3.35 TB/s bandwidth. Consumer GPUs use GDDR6X at ~1 TB/s. The bandwidth gap is why an H100 runs inference 2–3× faster than an RTX 4090 with similar TFLOPS — the compute is there, but the memory can't feed it fast enough.
Workflow example
Memory bandwidth is your inference speed limit. Calculate your theoretical max tok/s: bandwidth (TB/s) / model_size_per_token (GB). For a 7B Q4 model (~4 GB): 1 TB/s / 0.004 GB = 250,000. Each token requires reading ~4 GB of weights. Real-world is ~50% of theoretical due to overhead. An M2 Ultra (800 GB/s bandwidth) achieves ~100 tok/s on 7B Q4. An RTX 4090 (1 TB/s) hits ~120 tok/s.