01. What is an Embedding?

Chapter 1 of 18 · 20 min

KEY INSIGHT

An embedding converts text into a list of 768+ numbers that capture semantic meaningâ€”documents with similar meaning have similar numbers. When you feed the sentence "The cat sat on the mat" into an embedding model, you do not get back text. You get back a vector: a list of floating-point numbers. ```python from sentence_transformers import SentenceTransformer model = SentenceTransformer('all-MiniLM-L6-v2') # This produces a list of 384 numbers embedding = model.encode("The cat sat on the mat") print(f"Vector dimension: {len(embedding)}") print(f"First 10 values: {embedding[:10]}") ``` Output: ``` Vector dimension: 384 First 10 values: [ 0.02931074 -0.01298371 0.06068995 0.02664177 0.00397757 0.02698916 0.0092486 -0.04706898 0.02984722 -0.02698026] ``` These numbers mean nothing to humans, but they encode semantic relationships. The sentence "A feline rested on the rug" produces a vector close to the one aboveâ€”different words, similar meaning, similar numbers. The distance between two vectors tells you how semantically similar the underlying text is. You measure this with cosine similarity or Euclidean distance: ```python import numpy as np def cosine_similarity(a, b): return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)) sentence1 = model.encode("The cat sat on the mat") sentence2 = model.encode("A feline rested on the rug") sentence3 = model.encode("Quantum physics is fascinating") print(f"Cat vs Cat (similar): {cosine_similarity(sentence1, sentence2):.4f}") print(f"Cat vs Physics (dissimilar): {cosine_similarity(sentence1, sentence3):.4f}") ``` Output: ``` Cat vs Cat (similar): 0.7421 Cat vs Physics (dissimilar): 0.0923 ``` The model learned that cats and felines are related concepts. Physics and cats are not. Embeddings turn the fuzzy problem of "meaning" into the precise problem of "geometric distance." This makes search mathematical instead of lexical.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.

EXERCISE

Generate embeddings for three sentences of your choice. Calculate pairwise cosine similarities. Verify that semantically similar sentences score above 0.6 and dissimilar ones score below 0.3.