BGE (BAAI General Embedding)

BAAI's BGE (BAAI General Embedding) family is the canonical open-weight text embedding and reranking architecture, spanning three model generations built on the BERT→BGE→M3→Reranker lineage. The retroMAE phase is followed by contrastive learning on 1.2B text pairs covering retrieval, semantic similarity, and clustering tasks. BGE v1 achieved MTEB (Massive Text Embedding Benchmark) leadership at launch, scoring 63.8 on the MTEB retrieval average vs OpenAI text-embedding-ada-002's 58.6. [BGE-M3](/models/bge-m3) (January 2024, 568M parameters) introduced the M3 capability triplet: Multi-Lingual (100+ languages), Multi-Granularity (supporting input lengths from single words to 8192 tokens), and Multi-Functionality (dense retrieval, sparse lexical retrieval, and multi-vector retrieval from a single model). The architecture is a XLM-RoBERTa base encoder extended with: (1) a dense projection head (768-dim embeddings), (2) a sparse lexical head that outputs token-level importance weights (BM25-equivalent sparse vectors), and (3) a multi-vector head that produces token-level embeddings for ColBERT-style late-interaction retrieval. All three heads share the same encoder backbone — a single forward pass produces dense, sparse, and multi-vector representations simultaneously. BGE-Reranker-v2 (2024) shifted from encoder-only to encoder-decoder (T5-architecture) for cross-encoding. BGE-Reranker-v2-M3 (568M parameters, based on multilingual T5) performs pairwise relevance scoring: concatenating [query, document] into a single input and outputting a relevance score. This cross-encoding approach achieves 15–25% higher retrieval quality than bi-encoder dense retrieval at the cost of being ~100× slower (each query-document pair requires a separate forward pass vs bi-encoding where documents are pre-encoded). The operational pattern: BGE-M3 for first-stage retrieval (bi-encoder, fast, pre-computable document embeddings), BGE-Reranker-v2 for second-stage reranking (cross-encoder, high-quality, applied to top-100 candidates). BGE v2 (2026 expected) is anticipated to migrate from BERT/T5 architectures to a decoder-only LLM backbone (likely BAAI's Aquila model), unifying the embedding and reranking tasks into a single LLM-as-embedder architecture following the trend set by [E5-Mistral](/families/mistral) and GritLM.

BGE models are among the most locally-viable embedding families — the small model sizes and encoder-only architectures enable deployment on hardware ranging from CPU-only web servers to GPU clusters. BGE-small (24M parameters, ~100 MB ONNX) runs on CPU at ~50 sentences/second for single-sentence embedding — sufficient for small-scale semantic search on a laptop. BGE-base (110M, ~440 MB FP16) embeds at ~200 sentences/second on an [RTX 3060 12GB](/hardware/rtx-3060-12gb) with GPU acceleration and ~30 sentences/second CPU-only — the practical baseline for most local deployments. BGE-M3 (568M, ~2.2 GB FP16) is the quality/performance sweet spot. On an [RTX 4060 Ti 16GB](/hardware/rtx-4060-ti-16gb), BGE-M3 embeds at ~80 documents/second with batch size 32 — indexing a million-document corpus in ~3.5 hours. The 768-dimension embeddings consume ~3 GB per million vectors at FP16 — a billion-document index fits in 3 TB of RAM or SSD-backed vector storage. The [MacBook Pro 16" M4 Max](/hardware/macbook-pro-16-m4-max) with 128 GB unified memory can hold a ~40M document BGE-M3 index entirely in memory for sub-millisecond retrieval. BGE-Reranker-v2-M3 (568M, ~2.2 GB) has a fundamentally different local viability profile. Because reranking requires a forward pass per query-document pair (not a pre-computed embedding lookup), reranker throughput is measured in pairs/second, not documents/second. On an [RTX 4090](/hardware/rtx-4090), BGE-Reranker-v2-M3 processes ~200 query-document pairs/second at max-length 512 tokens. Reranking a top-100 candidate list for a single query takes ~0.5 seconds at batch-size-16. For 10 queries/second, this consumes ~5 seconds of GPU time per second of wall-clock — a single GPU can serve ~4–6 QPS of reranker traffic comfortably. For higher throughput, horizontal scaling (multiple GPUs with a load-balanced reranker pool) is straightforward because the reranker is stateless. The local embedding pipeline architecture is: BGE-M3 embeddings stored in [Qdrant](/tools/qdrant) or [pgvector](/tools/pgvector) for first-stage retrieval, BGE-Reranker-v2 for second-stage reranking. Both stages can run on the same GPU (since the reranker only activates after retrieval) or on separate hardware tiers (BGE-M3 on a cheap inference server, BGE-Reranker on a dedicated GPU for latency-sensitive ranking). This two-stage architecture is the standard local embedding deployment pattern and has been validated in production at scale by the BGE community.

BGE has dominated the MTEB (Massive Text Embedding Benchmark) leaderboard for open-weight models since August 2023. BGE-large-en-v1.5 scored 64.4 on the MTEB English average at launch, surpassing OpenAI text-embedding-ada-002 (58.6) and Cohere embed-v3 (62.3) — the first time open-weight embeddings matched commercial API quality. On the retrieval sub-score (the most operationally relevant MTEB dimension), BGE-large scored 62.3 vs ada-002's 56.7 — a 5.6-point gap that translates to ~15% improvement in recall@10 on standard retrieval benchmarks. [BGE-M3](/models/bge-m3)'s MTEB multilingual performance established a new ceiling. M3's key advantage: consistent performance across the three retrieval modes (dense, sparse, hybrid). The dense+sparse hybrid retrieval (combining dense cosine similarity and sparse BM25 scores) scored 67.8 on MTEB retrieval — a 3.6-point improvement over dense-only, the largest hybrid-boost delta recorded on MTEB for any model. On the BEIR benchmark suite (18 retrieval tasks covering biomedical, financial, fact-checking, and QA domains), BGE-large-v1.5 averaged 0.527 NDCG@10 vs the previous best open-weight model's 0.498. BGE-M3 improved this to 0.543 — competitive with the best commercial API embeddings evaluated on BEIR. The consistent finding: BGE models generalize well across retrieval domains without task-specific fine-tuning, a crucial property for production retrieval systems that serve diverse query types. [BGE-Reranker-v2-M3](/models/bge-reranker-v2-m3)'s benchmark story is about the cross-encoding quality ceiling. On the MTEB reranking average, BGE-Reranker-v2-M3 scored 69.2 vs the second-place open-weight reranker 65.1 and the commercial Cohere Rerank v3's 67.8. This bi-encoder→cross-encoder quality ladder is the canonical retrieval architecture that BGE established as the open-weight standard. On the CMTEB (Chinese MTEB) benchmark, BGE-M3 holds the open-weight lead at 67.1 vs the nearest competitor 64.8 — consistent with BAAI's Chinese-language optimization. On the MIRACL multilingual retrieval benchmark, BGE-M3 scores 0.612 NDCG@10 (average across 18 languages) vs the next-best open-weight model's 0.578 — a 3.4-point gap that narrows to 1.1 points for English-only retrieval, indicating BGE-M3's multilingual advantage is concentrated in non-English languages.

BGE models are natively supported in [Transformers](/tools/transformers) (via the AutoModel pipeline), Sentence-Transformers (the standard Python embedding library), and FlagEmbedding (BAAI's official inference library). Sentence-Transformers is the practical runtime for most deployments — it provides a unified `SentenceTransformer` class that loads any BGE variant, handles tokenization, pooling, and normalization, and outputs numpy arrays ready for cosine similarity computation. The Sentence-Transformers API is the lingua franca of the embedding ecosystem. FlagEmbedding (BAAI's official library) provides optimized inference for BGE models with built-in support for the M3's dense/sparse/multi-vector outputs and the Reranker's cross-encoding scoring. FlagEmbedding's BGE-M3 inference is ~15–20% faster than Sentence-Transformers on identical hardware due to optimized attention implementation and batched sparse-vector computation. vLLM pooling mode provides continuous batching, prefix caching for repeated system prompts, and the OpenAI /v1/embeddings API format — making BGE embeddings a drop-in replacement for the OpenAI embeddings API in any application that uses it. vLLM embedding throughput on an [A100 80GB](/hardware/nvidia-a100-80gb-sxm) with BGE-M3 at batch-128: ~3,000 embeddings/second — sufficient for indexing 10M documents in under an hour. [Ollama](/tools/ollama) does not natively support BGE embedding models (Ollama currently supports only text-generation models). For local embedding with a consumer-friendly interface, [llama.cpp](/tools/llama-cpp) provides BERT-based embedding support via the `llama-embedding` binary — but BGE-M3's XLM-RoBERTa architecture is not yet supported in llama.cpp. The practical local embedding runtime stack is: Sentence-Transformers + Flask/FastAPI for serving, or the [Qdrant](/tools/qdrant) FastEmbed library which bundles BGE-base and BGE-small as built-in embedding options. BGE-Reranker-v2 inference is via FlagEmbedding's `FlagReranker` class or Sentence-Transformers' `CrossEncoder` class. The reranker is stateless and embarrassingly parallel — deploy a pool of reranker instances behind a load balancer for horizontal scaling. The reranker's latency (20–50ms per pair on GPU) makes it suitable for post-retrieval reranking in real-time search with top-20 to top-100 candidates per query.

BGE quantization leverages the encoder-only architecture's robustness to weight precision reduction. BGE-M3 at INT8 (via ONNX Runtime or TensorRT) reduces the model size from 2.2 GB to ~800 MB with <0.5% MTEB score degradation — a practical tradeoff for embedding throughput-sensitive deployments. INT8 BGE-M3 on an [RTX 3060 12GB](/hardware/rtx-3060-12gb) embeds at ~120 documents/second (batch-32) vs ~80 documents/second at FP16 — the 1.5× throughput gain from INT8 tensor core acceleration outweighs the negligible quality cost. Post-training quantization of BGE-Reranker-v2 is more nuanced. INT8 Reranker-v2 shows ~1–2% NDCG degradation on MTEB reranking — acceptable for most use cases. Q4 introduces 5–8% degradation, primarily on queries requiring nuanced relevance distinctions (e.g., "mildly relevant" vs "highly relevant" documents). Q8 is the safe default for reranker quantization; INT8 is acceptable with validation on your specific retrieval domain. BGE-M3's sparse-lexical head is not quantizable without losing the sparse representation's BM25-equivalent quality. The practical approach: quantize the encoder backbone (INT8 or Q8) but keep the sparse-lexical head at FP16. This adds ~50 MB to the model size vs full-quantized but preserves the hybrid dense+sparse retrieval quality that makes BGE-M3 unique. ONNX Runtime provides the most mature BGE quantization path. The `optimum-cli` tool (Hugging Face) converts BGE models to ONNX with optional INT8 quantization in one command. ONNX INT8 BGE-base (~200 MB) achieves 500+ embeddings/second on CPU — viable for edge and embedded deployment where GPU is unavailable. The ONNX runtime's CPU execution provider (optimized for x86 and ARM) makes BGE embeddings feasible on Raspberry Pi 5-level hardware at 10–50 embeddings/second — sufficient for small-scale local semantic search (thousands of documents). Binary/pq (product quantization) of BGE embeddings (quantizing the output vectors, not the model weights) is standard for large-scale vector storage. 768-dimension FP16 embeddings → 768-dimension INT8 (SQ quantization) reduces storage by 4× with <0.2% recall@10 degradation. 16×6-bit PQ reduces storage by 16× with ~3–5% recall@10 degradation. The embedding quantization regime depends on retrieval quality requirements and storage budget — BGE embeddings are resilient to post-hoc quantization at standard compression ratios for vector databases.

BGE's embedding ecosystem rivals [Whisper's](/families/whisper) STT ecosystem in maturity and breadth. The Sentence-Transformers library (400+ BGE-based models on Hugging Face) is the PyTorch of text embeddings — it's the standard integration layer across vector databases, search frameworks, and RAG pipelines. Every major vector database ([Qdrant](/tools/qdrant), [pgvector](/tools/pgvector), Milvus, Weaviate, Chroma, Pinecone, Elasticsearch, OpenSearch) has dedicated BGE integration documentation and recommended embedding configurations. The BGE model hub on Hugging Face is comprehensive: BAAI's official models (BGE-small/base/large/M3/Reranker), community fine-tunes for specific languages and domains, ONNX/OpenVINO/TensorRT-optimized variants, and quantized GGUF/INT8 versions. The model card quality is above-average for the embedding ecosystem — each BAAI model card includes MTEB benchmark tables, usage examples in Sentence-Transformers and FlagEmbedding, and hardware requirements. The RAG (Retrieval-Augmented Generation) ecosystem has standardized on BGE as the default open-weight embedding model. LlamaIndex and LangChain both use BGE-small/default as their starter embedding model in quickstart tutorials. The "BGE-M3 → vector DB → top-k retrieval → BGE-Reranker → LLM" pipeline is the canonical local RAG architecture, documented across dozens of tutorials and implemented in open-source projects (Danswer, Quivr, Kotaemon, Verba). When a developer searches "how to build local RAG", the answer is overwhelmingly BGE-family-centric. The ecosystem gap is the embedding-reranker tightly-coupled deployment pattern. Each team builds custom middleware — there's no dominant "RAGOps" framework for embedding pipeline management. This is the next infrastructure layer that the BGE ecosystem needs to develop to match the model-quality maturity with operational maturity. The FlagEmbedding library (BAAI's official toolkit) fills the research-to-production gap: it provides training code, evaluation harnesses, and optimized inference for all BGE models. The library's `FlagModel`, `FlagReranker`, and `FlagLLMReranker` classes provide a unified API across the embedding→reranking→LLM-as-reranker pipeline. FlagEmbedding's active maintenance (weekly commits, rapid model-release support) makes it the most reliable integration surface for BGE in production.

BGE fine-tuning is accessible and well-documented. The FlagEmbedding library provides training code for contrastive learning (the standard embedding fine-tuning paradigm) with hard-negative mining, multi-task training, and data augmentation. Fine-tuning BGE-M3 on a domain-specific retrieval dataset (e.g., 10,000 query-document pairs with relevance labels from your organization's search logs) requires 16–24 GB VRAM and 1–4 hours on an [RTX 4090](/hardware/rtx-4090) using LoRA rank-16 on the encoder's attention query/value projections. This domain adaptation typically yields 5–15% recall@10 improvement on in-domain retrieval tasks. BGE's data preparation pipeline is the fine-tuning bottleneck, not the training infrastructure. This data curation step is 3–5× more time-consuming than the actual fine-tuning. The FlagEmbedding library provides LLM-based synthetic data generation (using GPT-4 or [Llama 3.3 70B](/models/llama-3-3-70b) to generate query-document pairs from your document corpus) that reduces the data preparation burden — but synthetic-data quality must be validated against real query patterns. BGE-Reranker-v2 fine-tuning is more GPU-intensive than bi-encoder fine-tuning. Cross-encoder training processes pairs rather than individual documents, making the effective batch size proportional to pair count × sequence length. Fine-tuning Reranker-v2 on 10,000 query-document pairs with relevance labels requires 24–32 GB VRAM (batch size 8 with gradient accumulation) and 4–8 hours on an [RTX 5090](/hardware/rtx-5090). The reranker domain adaptation is typically worth 10–20% NDCG improvement on in-domain retrieval — a higher per-fine-tune-hour ROI than bi-encoder fine-tuning because the reranker directly addresses the relevance-distinction task. Community fine-tunes cluster around: financial document retrieval (SEC filings, earnings reports), medical literature retrieval (PubMed, clinical trials), legal document retrieval (case law, contracts), code search (natural-language queries → code snippets), and multilingual domains under-served by BGE-M3's 100-language coverage (e.g., specific Indic language families, African languages). The BAAI organization on Hugging Face highlights and features high-quality community fine-tunes, creating a virtuous cycle of community contribution.

BGE-M3's multi-vector (ColBERT) retrieval mode is powerful but introduces an exponential storage cost. While dense embeddings consume 768 floats per document (~3 KB at FP16), the multi-vector representation consumes 768 × N floats where N is the number of tokens in the document (~200–500 KB for a typical paragraph). A million-document index at multi-vector resolution consumes 200–500 GB — 100× the dense embedding storage. For most deployments, dense retrieval (with optional sparse for hybrid scoring) is the practical mode — multi-vector retrieval is reserved for high-value, small-scale retrieval where latency is not a constraint. The embedding dimensionality mismatch between BGE model versions creates migration friction. BGE-base/large output 768-dimensional embeddings; BGE-M3 outputs 1024-dimensional dense embeddings (the sparse and multi-vector heads have separate dimensionalities). Switching from BGE-base to BGE-M3 requires re-indexing the entire document corpus — a 3.5-hour process for 1M documents on a single GPU. For production systems with live indices, plan for a zero-downtime migration: deploy BGE-M3 alongside BGE-base, populate a new vector database index, validate retrieval quality, then switch the query path — all while serving from the old index. BGE-Reranker-v2's inference latency compounds with document length and candidate count. Reranking 100 candidates with average document length of 512 tokens takes ~0.5s on an [RTX 4090](/hardware/rtx-4090) (single query). At 10 QPS, reranker latency dominates the retrieval pipeline (bi-encoder retrieval is 5–10ms, reranker is 100–500ms). For latency-sensitive applications (search-as-you-type, real-time chatbots), limit the reranker candidate set to top-10 or top-20 rather than top-100 — the quality gain from reranking 10→20 candidates is ~2–3% NDCG, while 20→100 yields ~1–2% additional gain at 5× the latency cost. The hybrid dense+sparse retrieval mode requires a vector database that supports hybrid scoring. [Qdrant](/tools/qdrant) provides native dense+sparse hybrid retrieval with configurable fusion (RRF or linear combination). [pgvector](/tools/pgvector) supports dense and sparse as separate indexes — hybrid scoring must be implemented in the application layer with post-retrieval fusion. Milvus and Weaviate provide native hybrid retrieval.