Dense Retrieval

Dense retrieval finds documents by computing cosine similarity (or dot product) between learned vector embeddings of the query and each document. Distinct from sparse retrieval (BM25, TF-IDF) which uses lexical token-frequency features.

Dense embeddings capture semantic similarity — a query about "automobile insurance" matches documents about "car coverage" even with no shared tokens. The cost: requires an embedding model and a vector index (FAISS, HNSW, IVF).

For local RAG, common embedding models are BGE-M3, Snowflake Arctic, and Qwen3-Embedding. A 384–1024 dimension index over 100K chunks fits comfortably in 1–4 GB of RAM and queries in <10 ms with HNSW.

Building a local RAG system over a company's internal wiki (roughly 50K chunks), an operator embeds each chunk with BGE-M3 at 1024 dimensions, producing a FAISS HNSW index that fits in under 1 GB of RAM and returns top-k results in single-digit milliseconds on CPU. Dense retrieval correctly surfaces a wiki page titled "Vehicle Reimbursement Policy" when a user asks "how do I get gas money back," despite zero token overlap — something BM25 alone would miss entirely. The tradeoff shows up on a follow-up query for an internal ticket ID like "INFRA-4521": the embedding model has never seen that token pattern in training and retrieves semantically related-but-wrong tickets instead, which is exactly the failure mode that pushes teams toward hybrid retrieval.