LangChain

LangChain is a Python/TypeScript framework for building applications that chain together LLM calls, external data sources, and tools. It provides abstractions like Chains, Agents, and Retrieval-Augmented Generation (RAG) pipelines. Operators encounter LangChain when they need to orchestrate multi-step workflows—e.g., fetching documents from a vector store, passing them as context to a local model, then formatting the output. LangChain supports local models via integrations with Ollama, Hugging Face, and llama.cpp, but its default design assumes cloud APIs, so running locally requires explicit configuration of endpoints and model names.

LangChain's core components include: (1) Models—wrappers around LLM APIs or local runtimes; (2) Prompts—templating and formatting; (3) Chains—sequences of calls (e.g., prompt → LLM → output parser); (4) Agents—LLMs that decide which tools to call; (5) Memory—state persistence across interactions. For local AI, operators often use LangChain's OllamaLLM or HuggingFacePipeline classes to point at a local model. However, LangChain's heavy abstraction layer adds latency and complexity compared to direct llama.cpp or vLLM usage. Many operators find LangChain useful for prototyping RAG systems but switch to lighter frameworks (e.g., raw llama.cpp server + custom scripts) for production. LangChain also introduced LangSmith for tracing and LangServe for deployment, but these are cloud-oriented.

An operator running Ollama with Llama 3.1 8B can use LangChain to build a RAG chain: from langchain_community.llms import Ollama; llm = Ollama(model="llama3.1:8b"). Then define a prompt template and a vector store (e.g., Chroma) to retrieve relevant chunks. The chain calls the local model at ~40 tok/s on an RTX 4090. Without LangChain, the same workflow requires manually handling embeddings, retrieval, and prompt formatting.

In a typical LangChain workflow, an operator writes a Python script that imports langchain and langchain_community. They instantiate an Ollama LLM, create a RetrievalQA chain with a Chroma vector store, and run chain.invoke("query"). The framework handles prompt construction, context injection, and output parsing. If the local model is slow, the operator may adjust num_predict or switch to a smaller quantized model.