Embedding (Vector Embedding)

An embedding is a fixed-length vector representation of text, image, or other input — typically 384-3072 dimensions — where semantic similarity corresponds to vector distance. "Cat" and "kitten" land closer in embedding space than "cat" and "airplane".

Embeddings are the backbone of semantic search and RAG. To find documents relevant to a query, you embed both the query and your document chunks, then retrieve the chunks with the smallest cosine distance. This works because the embedding model has been trained to put similar meanings close together.

For local-only RAG: BGE-large (1024-dim), E5-large (1024-dim), or nomic-embed-text-v1.5 (768-dim) all work well, and all run on a 4GB GPU. You don't need a frontier embedding model — the gap between BGE and OpenAI's text-embedding-3 is much smaller than the gap between an 8B and a frontier LLM.

Building a local RAG pipeline over a 50,000-document internal wiki, you embed every chunk once with nomic-embed-text-v1.5 (768-dim) on a spare RTX 3060, storing vectors in a local vector database. At query time you embed the user's question with the same model and retrieve the nearest chunks by cosine distance — the whole embedding pass for a query is fast even on modest hardware, since the model is tiny compared to your generation LLM. A common mistake operators make: switching embedding models mid-project without re-embedding the corpus. Vectors from BGE-large and nomic-embed-text live in different geometric spaces, so mixing them silently degrades retrieval quality with no error message — just worse RAG answers that are hard to diagnose.