09. Multi-Document Summarization

Multi-document summarization synthesizes information across multiple sources into coherent, consolidated summaries. Unlike single-document tasks, cross-document aggregation must reconcile conflicting information, identify consensus positions, and avoid redundant coverage of shared content.

Conflict detection identifies contradictory claims across source documents. When sources disagree on facts, summarization strategies range from neutral presentation acknowledging uncertainty to preference weighting based on source credibility. System prompt engineering must specify conflict handling policies.

from typing import List, Dict
import ollama

def multi_doc_summarize(documents: List[str], model: str = "llama3") -> Dict:
    # Stage 1: Individual document processing
    doc_summaries = []
    for i, doc in enumerate(documents):
        prompt = f"""Summarize this document in 3-5 sentences.
        Focus on key facts, claims, and conclusions.
        
        Document {i+1}:
        {doc}
        
        Summary:"""
        response = ollama.generate(model=model, prompt=prompt)
        doc_summaries.append({
            'index': i,
            'summary': response['response']
        })
    
    # Stage 2: Cross-document synthesis
    synthesis_prompt = f"""Synthesize these {len(documents)} document summaries 
    into a unified multi-document summary.
    
    Requirements:
    - Consolidate overlapping information
    - Preserve source diversity where perspectives differ
    - Flag contradictory claims with source attribution
    - Present consensus positions prominently
    - Note areas where sources add unique information
    
    Summaries:"""
    
    for d in doc_summaries:
        synthesis_prompt += f"\n\nDocument {d['index']+1}: {d['summary']}"
    
    synthesis_prompt += "\n\nUnified Summary:"
    
    unified = ollama.generate(model=model, prompt=synthesis_prompt)
    
    return {
        'individual_summaries': doc_summaries,
        'unified_summary': unified['response']
    }

def cross_doc_entity_tracking(documents: List[str], model: str = "llama3") -> Dict:
    """Track entities across multiple documents for relation synthesis."""
    prompt = """Extract named entities and track their appearances across documents.
    Identify relationships that span multiple documents.
    
    Documents: """ + "\n---\n".join(documents)
    
    response = ollama.generate(model=model, prompt=prompt)
    return parse_entity_relations(response['response'])

Hierarchical summarization processes documents at multiple levels. Initial pass extracts entity mentions and key claims. Intermediate aggregation identifies document clusters sharing topics. Final synthesis constructs coherent narrative from cluster summaries. This pyramid approach scales to document collections impractical for single-pass processing.

Temporal reasoning addresses document collections spanning different time periods. Summaries must distinguish current information from historical context, flag information that may have aged out, and indicate when evidence supersedes earlier claims. Temporal tagging in source documents assists priority determination.

Source attribution in multi-document summaries preserves accountability. Citations linking summary claims to source documents enable verification and allow readers to explore source context. Attribution styles range from inline references to footnotes to hyperlinked entity mentions.