Mistral
Mistral is a family of open-weight large language models (LLMs) developed by Mistral AI, known for their efficiency and strong performance relative to size. The family includes Mistral 7B (a 7-billion-parameter model), Mixtral 8x7B (a mixture-of-experts model with ~47B total parameters but only ~13B active per token), and Mistral Large (a larger proprietary model). Operators encounter Mistral models because they fit consumer hardware well: Mistral 7B runs on a 6 GB VRAM GPU at 4-bit quantization, and Mixtral 8x7B requires ~24 GB VRAM at 4-bit. Mistral models use a sliding-window attention mechanism, which reduces memory usage for long contexts.
Deeper dive
Mistral AI released Mistral 7B in September 2023, quickly gaining popularity for outperforming larger models like Llama 2 13B on many benchmarks. The architecture uses grouped-query attention (GQA) and a sliding window of 4096 tokens, meaning each token attends only to the previous 4096 tokens rather than the full sequence. This reduces memory and compute, especially for long documents. Mixtral 8x7B, released in December 2023, uses a mixture-of-experts (MoE) approach: it has 8 expert sub-networks, and for each token, a router selects the top-2 experts to process it. This yields higher capacity per token without proportional compute increase. Mistral models are available under the Apache 2.0 license (Mistral 7B, Mixtral 8x7B) or a commercial license (Mistral Large). They are supported by most local AI runtimes, including llama.cpp, Ollama, and Hugging Face Transformers.
Practical example
A user with an RTX 3060 (12 GB VRAM) can run Mistral 7B at Q4_K_M quantization, which uses ~4.5 GB VRAM, leaving room for a 4K context. The same user cannot run Mixtral 8x7B at Q4 because it requires ~24 GB VRAM; they would need to offload layers to system RAM, dropping tokens/sec from ~30 to ~5. On an Apple M2 Max with 64 GB unified memory, Mixtral 8x7B runs entirely in memory at Q4 at ~20 tok/s using MLX.
Workflow example
To run Mistral 7B with Ollama: ollama pull mistral downloads ~4.1 GB (Q4_0). Then ollama run mistral loads it into VRAM. For Mixtral 8x7B: ollama pull mixtral:8x7b downloads ~26 GB. In llama.cpp, you can specify -ngl 35 to offload 35 layers to GPU. In LM Studio, select a Mistral model from the hub, adjust the GPU offload slider, and monitor VRAM usage in the status bar.
Reviewed by Fredoline Eruo. See our editorial policy.