15. Conversational UX

Chapter 15 of 24 · 15 min

KEY INSIGHT

Conversational interfaces mask complexity behind natural interaction—but local models expose that complexity when context windows hit limits or memory constraints trigger reloading. Your UX must handle these transitions gracefully within the conversation metaphor. The core problem: conversation history grew too large for local context. Cloud solutions handle this invisibly (server-side context management). Local solutions must make the choice explicit while keeping the conversation coherent. ```python # conversation_manager.py from dataclasses import dataclass, field from typing import List, Optional import json @dataclass class Message: role: str # 'user', 'assistant' content: str tokens_approx: int = 0 timestamp: float = 0 @dataclass class ConversationContext: messages: List[Message] = field(default_factory=list) current_tokens: int = 0 max_context_tokens: int = 4096 model_name: str = "" system_prompt_tokens: int = 0 def add_message(self, role: str, content: str) -> str: """Add message and return strategy for overflow handling.""" tokens = self._estimate_tokens(content) msg = Message(role=role, content=content, tokens_approx=tokens) overflow = self.current_tokens + tokens + self.system_prompt_tokens - self.max_context_tokens if overflow > 0: strategy = self._calculate_compaction_strategy(overflow) return strategy self.messages.append(msg) self.current_tokens += tokens return "added" def _estimate_tokens(self, text: str) -> int: """Rough token estimation (chars / 4 for English).""" return len(text.split()) # Word-based approximation def _calculate_compaction_strategy(self, overflow: int) -> str: """Determine how to handle context overflow.""" if overflow > self.max_context_tokens * 0.5: return "summarize_and_continue" elif self.current_tokens > self.max_context_tokens * 0.7: return "trim_oldest" else: return "summarize_oldest" def get_summarized_history(self, keep_messages: int = 2) -> List[Message]: """Return messages with older ones potentially summarized.""" if not self.messages: return [] result = [] summary_tokens = sum(m.tokens_approx for m in self.messages[:-keep_messages]) if summary_tokens > 0: result.append(Message( role="assistant", content=f"[Previous conversation summarized ({summary_tokens} tokens removed)]", tokens_approx=20 )) result.extend(self.messages[-keep_messages:]) return result ``` Users should know when summarization happens. A subtle notification ("Earlier conversation condensed for memory efficiency") maintains trust. The alternative—losing message history without warning—breaks the conversational contract. For truly local products, consider prompt compression techniques: replacing verbose reasoning with condensed summaries before injecting into context. This is operator-level work that directly impacts UX quality.

EXERCISE

Implement a conversation manager that detects when context approaches limit (80% utilized) and pre-emptively notifies users. Include a method to generate automatic conversation summaries that preserve key facts while reducing token count. Create a test scenario with 20 messages that exceeds context limits.