What this does

A RetrievalQA chain retrieves relevant documents from a vector store and passes them as context to an LLM for answering. Adding source tracking lets you cite which documents contributed to each answer.

Steps

Create the vector store retriever. Set a retriever with a reasonable k value.

from langchain_community.vectorstores import Chroma
from langchain_community.embeddings import OllamaEmbeddings

embeddings = OllamaEmbeddings(model="nomic-embed-text")
vectorstore = Chroma(embedding_function=embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 3})

Build a prompt that asks for sources. Instruct the LLM to cite source metadata.

from langchain.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_template("""
Answer the question based on the context below. For each fact, cite the source.

Context:
{context}

Question: {question}

Answer with citations:""")

Create the RetrievalQA chain with source tracking. Use RetrievalQAWithSourcesChain or a custom chain.

from langchain.chains import RetrievalQAWithSourcesChain
from langchain_ollama import ChatOllama

llm = ChatOllama(model="llama3.2", temperature=0)

chain = RetrievalQAWithSourcesChain.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=retriever,
    return_source_documents=True
)

Invoke and inspect sources.

result = chain.invoke({"question": "What are the Q4 2025 financial results?"})

print("Answer:", result["answer"])
print("Sources:", result["sources"])
for doc in result["source_documents"]:
    print(f"  - {doc.metadata.get('source', 'unknown')}")

Alternative: custom chain with manual source extraction.

from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain

combine_docs_chain = create_stuff_documents_chain(llm, prompt)
rag_chain = create_retrieval_chain(retriever, combine_docs_chain)

result = rag_chain.invoke({"question": "What is RAG?"})
print(result["answer"])
for doc in result["context"]:
    print(f"Source: {doc.metadata.get('source', 'unknown')}")

Verification

python -c "
# Test with simple known document
from langchain_community.vectorstores import Chroma
from langchain_ollama import ChatOllama
vs = Chroma(...)
retriever = vs.as_retriever(k=1)
result = retriever.invoke('test')
print(f'Retrieved {len(result)} docs')
# Expected: Retrieved 1 docs
"

Common failures

Empty context makes the LLM hallucinate. If retrieval returns zero docs, the LLM invents an answer. Add a fallback check.
Source metadata missing. Documents indexed without source metadata field. Always include it during indexing.
Chain type mismatch. RetrievalQAWithSourcesChain requires return_source_documents=True to access individual docs.
Version mismatch - The installed package or runtime differs from the command shown; check the version first and rerun the smallest verification command.
Local environment drift - Another service, virtual environment, model, or path is being used; print the active binary path and configuration before changing the guide steps.

Related guides

How to Create Basic LangChain Chains with LCEL
How to Apply Metadata Filters to Reduce Search Space