RunLocalAI
intel
GPU
96GB VRAM
workstation

Intel Gaudi 2

Previous-gen Habana accelerator. 96GB HBM2e.

Released 2022

Overview

Previous-gen Habana accelerator. 96GB HBM2e.

Specs

VRAM96 GB
Power draw600 W
Released2022
MSRP$8000
Backends

Models that fit

Open-weight models small enough to run on Intel Gaudi 2 with usable context.

Llama 3.1 8B Instruct
8B · llama
Qwen 3 30B-A3B
30B · qwen
Qwen 2.5 Coder 32B Instruct
32B · qwen
Qwen 3 32B
32B · qwen
Gemma 4 31B Dense
31B · gemma
Qwen 3 8B
8B · qwen
DeepSeek R1 Distill Qwen 32B
32B · deepseek
Gemma 4 26B MoE
26B · gemma

Frequently asked

What models can Intel Gaudi 2 run?

With 96GB VRAM, the Intel Gaudi 2 runs 70B models in 4-bit quantization, plus everything smaller. See the model list below for tested combinations.

Does Intel Gaudi 2 support CUDA?

Intel Gaudi 2 does not support CUDA. Use Vulkan-compatible tools (llama.cpp Vulkan backend) or check vendor-specific runtimes.

Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify hardware specifications.