01. Part 1 Recap

Chapter 1 of 22 · 20 min

KEY INSIGHT

Part 1 built the ingestion-to-generation pipeline; Part 2 optimizes every stage of that pipeline for production. This chapter consolidates the foundational concepts from Part 1 that the rest of this course builds upon. If any concept feels unfamiliar, return to Part 1 before continuing. ### Vector Retrieval Pipeline A basic RAG pipeline works like this: documents get chunked, each chunk gets embedded into a dense vector, chunks get stored in a vector database, user queries get embedded using the same model, and the database returns the k-nearest chunks by cosine similarity. Those chunks get injected into an LLM prompt, and the LLM generates an answer. ```python from langchain_community.vectorstores import Chroma from langchain_openai import OpenAIEmbeddings # Basic retrieval setup from Part 1 embeddings = OpenAIEmbeddings(model="text-embedding-3-small") vectorstore = Chroma.from_documents( documents=chunks, embedding=embeddings, persist_directory="./chroma_db" ) # Query execution query_embedding = embeddings.embed_query(user_query) results = vectorstore.similarity_search_by_vector( query_embedding, k=10 ) ``` This pipeline has a hard ceiling on accuracy. The embedding model makes a one-time decision about what semantic content each chunk represents. Retrieval then just does nearest-neighbor search. Errors compound: a suboptimal chunk may contain relevant information but with distracting context, or the query may use different vocabulary than the chunk. ### Chunking Strategies Chunk size affects retrieval quality. Chunks too small lose context; chunks too large dilute relevance. Hierarchical chunking stores parent chunks for context while using child chunks for retrieval. Semantic chunking groups related content even if it crosses boundaries that would otherwise be arbitrary. ```python # From Part 1: hierarchical chunking def hierarchical_chunk(text, parent_size=1000, child_size=200): parent_chunks = text_chunker(text, chunk_size=parent_size) chunks_with_metadata = [] for i, parent in enumerate(parent_chunks): # Split parent into contextually coherent children children = semantic_split(parent, chunk_size=child_size) for child in children: chunks_with_metadata.append({ "content": child, "parent_idx": i, "parent_content": parent }) return chunks_with_metadata ``` ### Embedding Models Embedding models transform text into vectors that capture semantic meaning. The choice of model affects both retrieval accuracy and latency. Text-embedding-3-small offers good quality at low cost. BGE models provide strong open-source alternatives. Commercial models like Cohere and Voyage offer specialized optimizations. ### Limitations of Basic Retrieval Three failure modes dominate basic retrieval: **1. Vocabulary Mismatch.** User asks about "vehicle insurance" but documents say "auto coverage." Dense embeddings handle synonyms reasonably well but struggle with domain-specific terminology that wasn't well-represented during training. **2. Precision vs. Recall Tension.** Setting k=5 retrieves too few chunks when relevant information spans multiple sections. Setting k=50 retrieves too many, drowning relevant content in noise. **3. Ranking vs. Relevance Mismatch.** Cosine similarity in embedding space doesn't perfectly correlate with task-specific relevance. The chunk most semantically similar to the query may not be the chunk most useful for answering it. The following chapters address each of these failure modes with targeted techniques.

EXERCISE

Run your Part 1 RAG pipeline on three test queries. For each, note whether the failure was vocabulary mismatch, precision/recall tension, or ranking mismatch. Rank them by frequency.