16. Prompt with Retrieved Context

The prompt determines how the model uses retrieved context. Even excellent retrieval fails with poorly designed prompts. This chapter covers prompt engineering for RAG systems.

Basic RAG Prompt Template

RAG_PROMPT_TEMPLATE = """You are a helpful assistant that answers questions based ONLY on the provided context.

CONTEXT:
{context}

INSTRUCTIONS:
1. Answer the question using ONLY the context provided above.
2. If the context does not contain enough information to fully answer the question, say "Based on the provided context, I cannot fully answer this question."
3. Do NOT use any knowledge outside the provided context.
4. Quote relevant parts of the context when answering.

QUESTION: {question}

ANSWER:"""

This template includes guardrails preventing the model from hallucinating information not present in context. The "DO NOT use any knowledge outside" instruction significantly reduces fabrication.

Chain-of-Thought for Complex Queries

Complex analytical questions benefit from step-by-step reasoning:

ANALYTICAL_RAG_PROMPT = """You are an analyst answering questions based ONLY on the provided context.

CONTEXT:
{context}

Analyze the following question step by step:
"{question}"

Think step by step:
1. What information does the context provide that is relevant to this question?
2. How does this information combine to form an answer?
3. What specific parts of the context support each part of the answer?

Based on your analysis, provide a final answer:
"""

response = llm.generate(
    prompt=ANALYTICAL_RAG_PROMPT,
    context=assembled_context,
    question=user_question,
    temperature=0.3  # Lower temperature for factual tasks
)

Lower temperature (0.2-0.4) reduces creative liberties and keeps responses grounded in provided context.

Citation-Based Prompts

Force explicit citations to improve answer verifiability and reduce hallucination:

CITATION_PROMPT = """Answer the question using ONLY the provided context. Cite sources explicitly.

CONTEXT:
{context}

QUESTION: {question}

Format your answer as follows:
Answer: [Your answer here]
Sources: [Source 1], [Source 2]

Example:
Answer: The error occurs when memory exceeds 512MB.
Sources: troubleshooting.md (line 45), error_codes.md (line 12)
"""

The citation format forces the model to explicitly reference sources, making it easier to verify answers and identify hallucination.

System Prompt for Source Attribution

SYSTEM_PROMPT = """You are a technical support assistant. Your role is to:
1. Answer questions using ONLY information from the provided context
2. Always cite the specific source document when stating facts
3. Distinguish clearly between what the context says vs. what you know
4. If asked about topics not covered in context, state that the information is not available

Context is authoritative. Never contradict context even if it seems outdated."""

System prompts set persistent behavior expectations. The "never contradict context" instruction handles cases where the model knows more recent information than the retrieved documents contain.