Mistral 7B Instruct v0.3

Performance evidence

Performance statements in this entry have not been tied to a published, configuration-matched run. Treat speed figures as unverified estimates, not validated measurements. Compare the full configuration of a published benchmark before relying on throughput or latency.

Positioning

The model that defined local LLMs in 2023. Today, it's a benchmark baseline more than a working choice — every newer 7–8B model is meaningfully better while sitting in the same VRAM bracket. The Apache 2.0 license is its remaining real strength.

Strengths

• True Apache 2.0 license: no usage caps, no name-restrictions, no DUA. The most legally clean 7B in active use.
• Mature fine-tune ecosystem: thousands of derivatives, well-tested LoRA recipes, strong tooling support.
• Predictable runtime behavior: every runner has stable, well-debugged Mistral support — no surprises.

Limitations

• Instruction following lags Llama 3.1 8B: more frequent hallucinations on multi-step prompts, weaker JSON adherence.
• No system-prompt support in the v0.3 chat template — quirks the integration story for assistants and agent loops.
• Knowledge cutoff late-2023: noticeably stale on anything 2024+.

Historical performance claims (unverified) on RTX 4090

• Q4_K_M (4.4 GB): 100–120 tok/s decode, TTFT under 70 ms
• Q5_K_M (5.1 GB): 90–105 tok/s
• Q8_0 (7.7 GB): 75–88 tok/s

Should you run this locally?

Yes, for Apache-license-required commercial deployment, fine-tune base for novel domain adaptation, or as a regression baseline. No, for any general chat or assistant work — Llama 3.1 8B and Qwen 2.5 7B both beat it.

How it compares

• vs Llama 3.1 8B → Llama wins on instruction reliability, system-prompt support, and recency. The only reason to prefer Mistral is licensing.
• vs Qwen 2.5 7B → Qwen wins on knowledge breadth and multilingual; Mistral has the simpler license. Almost always pick Qwen unless license is the gating concern.
• vs Mistral Nemo 12B → Nemo replaces Mistral 7B v0.3 in the modern Mistral lineup — same Apache license, materially stronger model for ~50% more VRAM.
• vs Phi-3.5 Mini → comparable capability, Mistral uses ~2× the VRAM. Phi wins on efficiency.

Run this yourself

ollama pull mistral:7b-instruct-v0.3-q4_K_M
ollama run mistral:7b-instruct-v0.3-q4_K_M