02. Embedding Models Compared

Chapter 2 of 18 · 20 min

KEY INSIGHT

Model choice affects search quality more than database choiceâ€”MiniLM balances speed and quality for most local use cases. Embedding models vary in three dimensions: dimension size, quality, and speed. Larger dimensions capture more nuance but slow down similarity calculations. The most common open-source models: | Model | Dimensions | Quality Score | Speed (docs/sec) | |-------|------------|---------------|------------------| | all-MiniLM-L6-v2 | 384 | 68.4 | 14,000 | | paraphrase-Multilingual-MiniLM-L12-v2 | 384 | 70.1 | 10,000 | | all-mpnet-base-v2 | 768 | 72.7 | 2,800 | | bge-base-en-v1.5 | 768 | 73.4 | 3,200 | Benchmark from MTEB leaderboard (mass-text-embedding-benchmark). Higher quality scores indicate better retrieval performance. ```python from sentence_transformers import SentenceTransformer import time models = [ 'all-MiniLM-L6-v2', 'all-mpnet-base-v2', 'BAAI/bge-base-en-v1.5' ] test_corpus = ["Sample document number {}".format(i) for i in range(1000)] for model_name in models: model = SentenceTransformer(model_name) start = time.time() embeddings = model.encode(test_corpus, show_progress_bar=False) elapsed = time.time() - start docs_per_second = len(test_corpus) / elapsed print(f"{model_name}: {docs_per_second:.0f} docs/sec, shape {embeddings.shape}") ``` Expect output like: ``` all-MiniLM-L6-v2: 14235 docs/sec, shape (1000, 384) all-mpnet-base-v2: 2810 docs/sec, shape (1000, 768) BAAI/bge-base-en-v1.5: 3150 docs/sec, shape (1000, 768) ``` For local development and most production use cases, `all-MiniLM-L6-v2` is the practical choice. It encodes 14,000 documents per second on a modern laptop. If you need maximum quality and can tolerate 5x slower encoding, `bge-base-en-v1.5` is the better choice. Multilingual support matters if you index documents in multiple languages. The multilingual model handles English and 50+ other languages in the same vector spaceâ€”English "cat" and Spanish "gato" end up close together. ```python # Multilingual model usage multilingual = SentenceTransformer('paraphrase-multilingual-MiniLM-L12-v2') en_embedding = multilingual.encode("The cat") es_embedding = multilingual.encode("El gato") de_embedding = multilingual.encode("Die Katze") print(f"English-Spanish similarity: {cosine_similarity(en_embedding, es_embedding):.4f}") print(f"English-German similarity: {cosine_similarity(en_embedding, de_embedding):.4f}") ```

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.

EXERCISE

Benchmark all-MiniLM-L6-v2 against one other model. Time encoding 5000 short texts. Calculate the cost in seconds and estimate how long indexing 100,000 documents would take.