NVLink

NVLink is NVIDIA's proprietary GPU-to-GPU interconnect, used to bind multiple data-center GPUs into a coherent memory fabric. NVLink 4 (H100) runs at 900 GB/s bidirectional per link; multiple links per GPU stack to total bandwidth.

For local AI, NVLink matters when running multi-GPU tensor parallelism: a 70B model split across 2× RTX 3090s with NVLink hits significantly higher tok/s than the same setup over PCIe 4.0 (32 GB/s) because of the all-reduces between layers.

Consumer NVLink ended with the RTX 30 series. RTX 40 and 50 series have no NVLink — multi-GPU on consumer cards now relies on PCIe alone, which is the major bottleneck for tensor-parallel local inference.

An operator building a local 70B inference rig considers two used RTX 3090s versus two RTX 4090s. On paper the 4090s have more raw compute, but the 3090 generation supports NVLink, letting the pair form a coherent memory fabric for tensor-parallel splitting of the model across both cards. Running the same 70B split, the NVLink-connected 3090s show noticeably better scaling than the 4090 pair, which is stuck communicating over PCIe 4.0 at roughly 32 GB/s during the all-reduce steps between layers. Since NVIDIA dropped NVLink from RTX 40 and 50 series consumer cards entirely, this makes older dual-3090 builds a specific, still-relevant niche for anyone doing multi-GPU tensor parallelism on a budget rather than single large-VRAM cards.