ColPali v1.3
3B-parameter visual document retriever built on PaliGemma-3B using a ColBERT-style late-interaction objective. Encodes a PDF page as a grid of patch embeddings, skipping OCR/layout parsing entirely. Sets SOTA on the ViDoRe benchmark for visual document retrieval.
The clear winner for visual-document RAG when the corpus is PDF-heavy and full of charts/tables. Budget for the storage overhead of multi-vector indexes and pick your vector store deliberately — that's the only real friction.
Overview
3B-parameter visual document retriever built on PaliGemma-3B using a ColBERT-style late-interaction objective. Encodes a PDF page as a grid of patch embeddings, skipping OCR/layout parsing entirely. Sets SOTA on the ViDoRe benchmark for visual document retrieval.
Strengths
- SOTA on ViDoRe (~81 nDCG@5 on v1.x) — beats unstructured.io + BGE pipelines by 15+ points
- Skips OCR entirely — handles charts, tables, handwriting, low-quality scans natively
- Late-interaction (ColBERT-style) gives per-token precision while staying queryable
- MIT licensed, PaliGemma backbone — fully Apache/MIT downstream
- Indexes 1 page in <100ms on an RTX 3090
Weaknesses
- Multi-vector index: ~1030 embeddings per page — storage cost is ~100x dense retrievers
- Needs a late-interaction-capable store (Vespa, Qdrant multi-vector, plaid-x) — not a drop-in for FAISS
- Inherits PaliGemma's Gemma license restrictions on the base weights (LoRA adapter is MIT)
- Only the retriever — pair with a generative VLM (Qwen2.5-VL, GPT-4o, etc.) for answers
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| Q4_K_M | 1.7 GB | 3 GB |
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of ColPali v1.3.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run ColPali v1.3?
Can I use ColPali v1.3 commercially?
What's the context length of ColPali v1.3?
Source: huggingface.co/vidore/colpali-v1.3
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify ColPali v1.3 runs on your specific hardware before committing money.