InternVL 2.5 26B
InternVL 2.5 mid-tier — Shanghai AI Lab vision-language model with strong document and chart understanding.
Positioning
InternVL 2.5 26B is a dense vision-language model (VLM) from OpenGVLab, released under the permissive MIT license. With 26 billion parameters and a 32,768-token context window, it targets the consumer deployment class, fitting into 24GB VRAM at lower quantizations. This model is designed for document and chart understanding, making it a strong candidate for local multimodal tasks where licensing flexibility is a priority.
Strengths
- Permissive MIT License: Unlike many VLMs restricted to research, InternVL 2.5 can be freely used for commercial applications, including fine-tuning and deployment.
- Consumer-Friendly Quantization: At Q4_K_M (14.6 GB on disk) and Q3_K_M (12.7 GB), the model fits comfortably on a single 24GB GPU, with room for KV cache and overhead.
- Dense Architecture: As a dense 26B model, it avoids the complexity and memory overhead of mixture-of-experts (MoE) designs, simplifying deployment and inference.
- Document and Chart Focus: The model is specifically tuned for structured visual tasks, making it a practical choice for OCR, data extraction, and visual question answering on documents.
Limitations
- No Community Benchmarks Available: We do not have independently verified benchmark scores for this model. Published vendor metrics should be treated as best-case until confirmed by third-party testing.
- Large FP16 Footprint: At ~52 GB in FP16, the model requires significant VRAM for full-precision inference, limiting unquantized use to datacenter GPUs.
- Moderate Context Window: 32K tokens is adequate for many document tasks but may be restrictive for long-form video or multi-page document analysis.
- Niche Best-Use Case: While strong on documents and charts, the model may not generalize as well to other vision-language domains (e.g., natural images, video) without further evaluation.
What it takes to run this locally
At FP16, the model requires ~52 GB of disk space and roughly 52 GB of VRAM, placing it in the datacenter class. However, quantized versions dramatically reduce requirements:
- Q8_0: ~28 GB on disk, fits on a single 48GB workstation GPU (e.g., RTX 6000 Ada).
- Q4_K_M: ~14.6 GB on disk, fits on a single 24GB consumer GPU (e.g., RTX 4090) with ~30-50% additional overhead for KV cache and framework.
- Q3_K_M: ~12.7 GB on disk, also fits on 24GB GPUs with more headroom.
- Q2_K: ~8.5 GB on disk, fits on 12-16GB GPUs (e.g., RTX 3060/4070) but with expected quality loss.
For typical use with a 32K context, add ~30-50% to the model size for KV cache and framework overhead. A 24GB GPU running Q4_K_M is the recommended consumer setup.
Should you run this locally?
Yes if you need a permissively licensed VLM for commercial document/chart understanding and have a 24GB GPU (or can use Q3_K_M on smaller hardware). The MIT license and dense architecture make it a straightforward choice for local deployment.
No if your tasks require longer context (>32K), you need a general-purpose VLM for diverse visual inputs, or you cannot tolerate the quality loss from aggressive quantization (Q2_K). Also, if you lack community-verified benchmarks, you may prefer a model with more independent testing.
Catalog cross-links
- InternVL 2.5 78B – larger sibling with MoE architecture
- Qwen2-VL 72B – alternative VLM with 128K context
- LLaVA-NeXT 34B – another dense VLM with permissive license
Overview
InternVL 2.5 mid-tier — Shanghai AI Lab vision-language model with strong document and chart understanding.
How to run it
InternVL 2.5 26B is OpenGVLab's 26B vision-language model — the smaller sibling of InternVL 2.5 78B. 26B text backbone + InternViT vision encoder, designed for document understanding, OCR, and visual QA. Run at Q4_K_M via llama.cpp with llava-server for vision. Q4_K_M file size 15 GB (text) + ~3-5 GB (vision). Minimum VRAM: 16 GB — RTX 4080 (16GB) at Q4_K_M text-only, or Q3_K_M + vision. Recommended: RTX 4090 24GB at Q4_K_M + vision. Throughput: ~30-50 tok/s on RTX 4090 at Q4_K_M text-only; vision encoding adds ~1-3s per image. InternVL architecture — InternViT encoder is large (6B), making vision VRAM proportionally higher than Llama/Qwen vision models at the same text backbone size. Check llama.cpp InternVL 26B support — may differ from 78B support. Use for: document OCR, chart understanding, visual QA, UI screenshot analysis. Not for: text-only general chat (use standard 26B text model). Context: 32K advertised; practical with vision at Q4 on 24 GB is 4-8K. For larger vision models: InternVL 2.5 78B.
Hardware guidance
Minimum: RTX 3060 12GB at Q3_K_M + vision (tight). Recommended: RTX 4090 24GB at Q4_K_M + vision (8K context). VRAM math: 26B text at Q4 ≈ 15 GB. InternViT encoder: ~4-6 GB. KV cache at 8K: ~5 GB. Total with vision: ~24-26 GB. RTX 4090 24GB: Q4 + vision + 4K context — tight. Offload vision encoder activations for headroom. RTX 4080 16GB: Q3_K_M + vision at 4K. MacBook Pro M4 Max 36GB+: Q4 + vision at 5-10 tok/s. Cloud: A10 24GB at Q4_K_M + vision. InternViT is the bottleneck — budget 4-6 GB specifically for the vision encoder. AWQ-INT4 drops text to ~13 GB, helping VRAM fit.
What breaks first
- InternViT VRAM domination. The vision encoder is proportionally larger than the text backbone. At 26B text, the 6B vision encoder takes 25-30% of total VRAM — much higher ratio than Llama/Qwen vision models. 2. Multimodal GGUF scarcity. Pre-converted InternVL 26B GGUFs with vision are rare. You may need to convert from hf or use text-only. 3. Resolution sensitivity. InternViT's quality degrades sharply with low-resolution inputs. But high-res inputs spike vision encoder VRAM by 3-5 GB. Find the resolution sweet spot for your use case. 4. Tokenizer format. InternVL uses a custom vision+text tokenizer format. Standard llama.cpp llava may not handle InternVL's specific multimodal token embedding correctly. Validate vision outputs against reference.
Runtime recommendation
Common beginner mistakes
Mistake: Expecting InternVL 26B to have the same vision-to-text VRAM ratio as Llama 3.2 Vision. Fix: InternViT is ~6B — 20× larger than CLIP. Budget 4-6 GB for vision encoder alone. Your 16 GB GPU may not fit vision+text at Q4. Mistake: Using InternVL 26B vision projector with 78B GGUF. Fix: Different model sizes, different projectors. Match models exactly. Mistake: Assuming 26B = half the quality of 78B. Fix: The 26B is significantly weaker at complex visual reasoning. 78B is the recommendation for document understanding and OCR. 26B is the budget option. Mistake: Sending images without preprocessing. Fix: InternVL expects specific image preprocessing. Use the model's image processor or resize to the encoder's expected input size.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
- MIT license
- Strong on charts and documents
Weaknesses
- Smaller community than Qwen-VL
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| Q4_K_M | 16.0 GB | 20 GB |
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of InternVL 2.5 26B.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run InternVL 2.5 26B?
Can I use InternVL 2.5 26B commercially?
What's the context length of InternVL 2.5 26B?
Does InternVL 2.5 26B support images?
Source: huggingface.co/OpenGVLab/InternVL2_5-26B
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify InternVL 2.5 26B runs on your specific hardware before committing money.