NPU (Neural Processing Unit)
Definition pending
We've cataloged "NPU (Neural Processing Unit)" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
NPU (Neural Processing Unit) is a dedicated AI accelerator chip — separate from CPU and GPU. Found in newer laptop SoCs (Snapdragon X Elite: 45 TOPS NPU, Apple M4: 38 TOPS Neural Engine, Intel Meteor Lake: 11 TOPS NPU). They're optimized for low-power inference (<10W) of small models. Not suitable for LLMs > 3B params yet.
Workflow example
NPU usage is evolving fast. As of 2025: (1) Qualcomm NPU runs Llama 3.2 3B at ~20 tok/s on-device via ONNX/QNN, (2) Apple Neural Engine runs small models via Core ML, (3) Intel NPU is mainly for background tasks (camera effects, voice). For LLM operators: NPUs are future hardware, not current. Track the NPU TOPS ratings on /hardware/ but don't buy hardware specifically for NPUs yet — CUDA/Metal is still the practical standard.