15. Hardware Requirements
Video multimodal inference has distinct hardware requirements from image-based tasks. Memory bandwidth and frame buffer capacity often become bottlenecks before compute utilization reaches saturation.
For real-time 1080p video processing at 30fps, the raw input bandwidth exceeds 1.5 Gbps. When combined with intermediate feature maps and multiple model stages, peak memory throughput requirements can reach 10-20 GB/s. NVIDIA RTX series cards provide sufficient bandwidth for single-stream processing, but multi-camera systems quickly exhaust memory bus capacity.
# Check GPU memory bandwidth
nvidia-smi --query-gpu=mem.bandwidth.total --format=csv,noheader
# Monitor utilization during inference
nvidia-smi dmon -c 60 -s um
CPU requirements depend on preprocessing pipeline complexity. FFmpeg-based decode can saturate multiple CPU cores on high-resolution streams. ARM-based edge devices (Jetson AGX, Google Edge TPU) provide integrated video decode but have limited memory bandwidth that constrains model size.
Memory capacity becomes critical when buffering frames for temporal reasoning. A 30-frame buffer at 1080p RGB requires approximately 800MB. Storing intermediate activations for gradient computation during training multiplies this requirement by the batch size. For training, plan for 16-32GB GPU memory minimum; inference can operate with 8GB for many models.
Power consumption affects deployment viability. A full-featured multimodal video system might consume 300-500W continuously, making thermal management essential. Edge deployments require calculating total system power including cameras, storage, and networking equipment.
Local verification checkpoint
Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.
Profile a video inference pipeline and identify the bottleneck (compute, memory bandwidth, or CPU preprocessing). Use ncu (NVIDIA Nsight Compute) to generate a detailed timeline.