How to Implement Semantic Chunking with LangChain

What this does

Semantic chunking splits documents at natural topic or meaning boundaries rather than fixed character counts. The algorithm embeds sentences, measures cosine similarity between consecutive embeddings, and breaks the document whenever similarity drops below a threshold. This produces chunks that cohere around single ideas.

Steps

1. Confirm the embedding model is available.

   ollama pull nomic-embed-text
   

2. Import the semantic chunker.

   from langchain_experimental.text_splitter import SemanticChunker
   from langchain_ollama import OllamaEmbeddings
   
   embeddings = OllamaEmbeddings(model="nomic-embed-text")
   chunker = SemanticChunker(embeddings, breakpoint_threshold_type="gradient")
   

3. Load a document and split it semantically.

   from langchain_community.document_loaders import TextLoader
   
   loader = TextLoader("/data/article.txt")
   docs = loader.load()
   semantic_chunks = chunker.split_documents(docs)
   print(f"Produced {len(semantic_chunks)} semantic chunks")
   

4. Adjust sensitivity with a custom threshold.

   fine_chunker = SemanticChunker(
       embeddings,
       breakpoint_threshold_type="percentile",
       breakpoint_threshold_amount=50,
   )
   fine_chunks = fine_chunker.split_documents(docs)
   print(f"Fine-grained: {len(fine_chunks)} chunks")
   

Verification

python -c "
from langchain_experimental.text_splitter import SemanticChunker
from langchain_ollama import OllamaEmbeddings
e = OllamaEmbeddings(model='nomic-embed-text')
c = SemanticChunker(e)
print('SemanticChunker instantiated OK')
"
# Expected: SemanticChunker instantiated OK

Common failures

• ConnectionError on embed call. Ollama not listening on port 11434. Check with curl http://localhost:11434/api/tags.
• One monolithic chunk returned. Threshold too high. Lower breakpoint_threshold_amount or switch to "gradient" type.
• Slow on long documents. Per-sentence embedding is O(n). Batch sentences or pre-fetch embeddings into a cache.
• ImportError for langchain_experimental. Install with pip install langchain-experimental.
• Version mismatch - The installed package or runtime differs from the command shown; check the version first and rerun the smallest verification command.
• Local environment drift - Another service, virtual environment, model, or path is being used; print the active binary path and configuration before changing the guide steps.

Operator checkpoint

Before treating this as solved, write down the local runtime, model or package version, hardware/backend if relevant, and the verification output. This keeps the guide useful as a Will-It-Run style decision instead of a one-off command transcript.

Related guides

• use-recursive-character-text-splitter
• implement-hybrid-chunking-semantic-fixed