FAISS

FAISS (Facebook AI Similarity Search) is a C++/Python library for fast approximate nearest-neighbor search over dense vectors. The de-facto baseline for vector indexing — supports flat (exact), HNSW, IVF, PQ, and combinations.

For local RAG, FAISS is what's under the hood of Chroma, LangChain's default vector store, and many in-process embeddings setups. Index choice matters: HNSW is fast and accurate but memory-heavy; IVF-PQ trades recall for 10× smaller indexes.

For corpora under ~1M chunks, flat FAISS (exact nearest neighbor) is fast enough on CPU and avoids approximate-recall surprises. Beyond that, HNSW is the standard pick.

An operator building local RAG over 400K technical-doc chunks (embedded with a 768-dim model) starts with flat FAISS — IndexFlatIP — and gets sub-50ms query latency on CPU with zero recall loss, since brute-force cosine search over 400K vectors is cheap. Six months later the corpus grows to 8M chunks and flat search starts costing seconds per query; they switch to IndexHNSWFlat with M=32, efConstruction=200, trading a small amount of recall for order-of-magnitude faster lookups, but the index now eats several GB of RAM because HNSW stores the full-precision vectors plus graph edges. If RAM becomes the bottleneck instead of latency, IVF-PQ compresses vectors into codebooks first, shrinking the index by roughly 10x at the cost of coarser recall — a reasonable trade for a background retrieval step feeding a local dense-retrieval pipeline rather than a latency-critical one.