What this does

ConversationalRetrievalChain combines document retrieval with chat history, enabling multi-turn Q&A where the model remembers prior context while answering new questions from retrieved documents.

Steps

Set up memory and retriever. Use ConversationBufferMemory or ConversationSummaryMemory.

from langchain.memory import ConversationBufferMemory
from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import OllamaEmbeddings

embeddings = OllamaEmbeddings(model="nomic-embed-text")
vectorstore = Chroma(embedding_function=embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

memory = ConversationBufferMemory(
    memory_key="chat_history",
    return_messages=True,
    output_key="answer"
)

Create the conversational chain. Use ConversationalRetrievalChain with a condense-question prompt.

from langchain.chains import ConversationalRetrievalChain
from langchain_ollama import ChatOllama

llm = ChatOllama(model="llama3.2", temperature=0)

qa_chain = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=retriever,
    memory=memory,
    return_source_documents=True
)

Ask follow-up questions. The chain rephrases follow-ups in context of history before retrieval.

result1 = qa_chain.invoke({"question": "What is the company's revenue policy?"})
print(result1["answer"])

result2 = qa_chain.invoke({"question": "How does it compare to last year?"})
print(result2["answer"])
# The chain understands "it" refers to "revenue policy" from prior turn.

Inspect chat history. Verify what the chain remembers.

print(memory.chat_memory.messages)

Use summary memory for long conversations. Swap buffer for summary to avoid context overflow.

from langchain.memory import ConversationSummaryMemory

summary_memory = ConversationSummaryMemory(
    llm=llm,
    memory_key="chat_history",
    return_messages=True
)

Verification

python -c "
from langchain.memory import ConversationBufferMemory
m = ConversationBufferMemory(memory_key='chat_history', return_messages=True)
m.chat_memory.add_user_message('Hello')
m.chat_memory.add_ai_message('Hi there')
print(len(m.chat_memory.messages))
# Expected: 2
"

Common failures

Memory key mismatch. The chain expects chat_history but memory uses a different key. Verify the memory_key argument.
Retrieval ignores conversation context. The chain condenses the question before retrieval — ensure the condense question prompt works with your LLM.
Context window overflow. Long conversations + retrieved docs exceed the model's context limit. Switch to ConversationSummaryMemory.
Version mismatch - The installed package or runtime differs from the command shown; check the version first and rerun the smallest verification command.
Local environment drift - Another service, virtual environment, model, or path is being used; print the active binary path and configuration before changing the guide steps.

Related guides

How to Build RetrievalQA Chain with Sources
How to Implement Memory in LangGraph State