Advanced RAG — Chunking, Retrieval, Re-ranking

15. Context Optimization

Chapter 15 of 24 · 20 min

KEY INSIGHT

Raw retrieved chunks are often noisy; optimization filters and reorders context before generation. ### The Retrieval Noise Problem Vector similarity search returns chunks that are topically relevant but may include filler sentences, malformed code blocks, or off-topic tangents. Context optimization runs post-processing on retrieval results to improve their quality before feeding them to the answer-generating LLM. ### Techniques **Dedup**: Remove near-duplicate chunks that appear across multiple result slots. ```python from difflib import SequenceMatcher def removeNearDuplicates(chunks: list[dict], threshold: float = 0.85) -> list[dict]: deduped = [] for chunk in chunks: is_duplicate = False for kept in deduped: ratio = SequenceMatcher( None, chunk["text"], kept["text"] ).ratio() if ratio >= threshold: is_duplicate = True # Keep the one with higher relevance score if chunk.get("score", 0) > kept.get("score", 0): deduped.remove(kept) deduped.append(chunk) break if not is_duplicate: deduped.append(chunk) return deduped ``` **Noise Reduction**: Remove chunks below a relevance score threshold. ```python RELENCE_THRESHOLD = 0.65 def filterByRelevance(chunks: list[dict], threshold: float = RELENCE_THRESHOLD) -> list[dict]: return [c for c in chunks if c.get("score", 0) >= threshold] ``` **Diversity Boost**: Re-rank by MMR (Maximum Marginal Relevance) to penalize retrieving too many chunks from the same section. ```python import numpy as np def mmrRerank( chunks: list[dict], lambda_param: float = 0.5, top_k: int = 10 ) -> list[dict]: """ lambda_param: 0 = pure diversity, 1 = pure relevance. """ selected = [] remaining = chunks.copy() while len(selected) < top_k and remaining: if len(selected) == 0: selected.append(remaining.pop(0)) continue best_score = -float("inf") best_idx = None for i, chunk in enumerate(remaining): relevance = chunk.get("score", 0) # Max similarity to already selected chunks (diversity penalty) max_sim = max( cosineSimilarity(chunk["embedding"], s["embedding"]) for s in selected ) mmr_score = lambda_param * relevance - (1 - lambda_param) * max_sim if mmr_score > best_score: best_score = mmr_score best_idx = i if best_idx is not None: selected.append(remaining.pop(best_idx)) else: break return selected def cosineSimilarity(a: np.ndarray, b: np.ndarray) -> float: return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))) ``` ### Chaining Optimizations ```python def optimizeContext(chunks: list[dict], top_k: int = 10) -> list[dict]: # 1. Filter by relevance first filtered = filterByRelevance(chunks) # 2. Remove near-duplicates deduped = removeNearDuplicates(filtered) # 3. Re-rank with MMR for diversity reranked = mmrRerank(deduped, lambda_param=0.7, top_k=top_k) return reranked ``` ### Failure Modes Over-aggressive filtering can remove relevant low-scoring chunks that happen to be the only source for a specific fact. Always check for missing-entity scenarios where a fact appears only in a low-relevance chunk. Lambda tuning (MMR) often requires per-domain adjustment; a dataset collected from real queries is the most reliable tuning signal.

EXERCISE

Implement the optimization chain on your retrieval results. Evaluate answer quality change with a reference answer or LLM judge comparing filtered vs. unfiltered context. (15 min)