17. Prompt Chaining

Chapter 17 of 25 · 15 min

KEY INSIGHT

Chain reliability depends on output format consistency between steps, not on instructions within each step alone. ```python # Step 1: Extract structured information classifiction_prompt = """Extract key entities from this text. Output format: - Entity: [type] | [value] - Entity: [type] | [value] Text: {input_text}""" # Step 2: Validate extracted entities against source validation_prompt = """For each entity below, verify it appears verbatim in source text. If verified, mark [OK]. If not found, mark [NOT FOUND] and explain alternatives. Entities: {extracted_output} Source: {original_text}""" # Step 3: Generate output based on validated information synthesis_prompt = """Using these verified entities: {validated_output} Answer the question: {user_question}""" def run_chain(input_text, user_question): raw_extraction = model.generate(classifiction_prompt.format(input_text=input_text)) validation = model.generate(classification_prompt.format( extracted_output=raw_extraction, original_text=input_text )) # Only proceed with entities marked [OK] verified = parse_verified_entities(validation) final = model.generate(synthesis_prompt.format( validated_output=verified, user_question=user_question )) return final ``` **Failure mode:** Chain breaks when output format varies. Models are inconsistent in format adherence, especially with complex output schemas. A classification prompt specifying `{type: string, value: string}` may output `type=value`, `type = value`, or abandon the format entirely. ```python # Fallback technique: structure parsing around partial matching def extract_entity_fields(model_output): """Parse output that may deviate from strict schema.""" import re entities = [] for line in model_output.split('\n'): if ':' in line or '|' in line: # Handle various delimiters and spacing parts = re.split(r'[:|]', line, maxsplit=1) if len(parts) == 2: raw_type = parts[0].strip() raw_value = parts[1].strip() # Normalize type names entity_type = raw_type.lower().replace('entity', '').strip() entities.append({'type': entity_type, 'value': raw_value}) return entities ``` Chains longer than 4 steps exhibit error accumulation rates of 5–15% per step. Best practice limits chains to 3–4 steps with validation checkpoints between major transformations.

Prompt chaining connects sequential model calls where each step's output becomes the next step's input. The chain structure addresses problems too complex for single prompts while enabling intermediate verification points.

EXERCISE

Design a three-step chain for summarizing legal documents: (1) extract key terms and dates, (2) categorize clauses by type, (3) generate plain-language summary. Implement in code and test with one legal document, tracing where format inconsistencies appear.