14. UX for AI Products

Chapter 14 of 24 · 15 min

KEY INSIGHT

Local AI products require UX that communicates capability boundaries honestly. Users who expect cloud-like speed will blame your product for hardware they chose. Your interface must set accurate expectations while making local inference feel responsive. The challenge is latency perception. Even a fast local model (100ms response) feels different from streaming cloud output. Your UX needs to acknowledge this without making local feel inferior. The solution: show the value of privacy with the same visual weight you'd give to response speed. ```python # ui_progress_handler.py from dataclasses import dataclass from typing import Optional import time @dataclass class GenerationState: status: str # 'processing', 'streaming', 'complete', 'error' tokens_generated: int = 0 tokens_per_second: Optional[float] = None privacy_badge: bool = True # Local processing indicator elapsed_ms: int = 0 memory_usage_mb: float = 0 model_name: str = "" class LocalModelUXController: def __init__(self, model_manager): self.model_manager = model_manager self.start_time = None def on_start(self, prompt_tokens: int): """Called when generation begins.""" self.start_time = time.perf_counter() return GenerationState( status='processing', privacy_badge=True, model_name=self.model_manager.current_model ) def on_token(self, token: str) -> dict: """Called for each generated token (streaming simulation).""" if self.start_time: elapsed = (time.perf_counter() - self.start_time) * 1000 return { 'token': token, 'elapsed_ms': int(elapsed), 'privacy_indicator': '🔒 Local inference active' } return {'token': token} def on_complete(self, total_tokens: int) -> GenerationState: """Called when generation finishes.""" if self.start_time: elapsed = (time.perf_counter() - self.start_time) * 1000 tps = total_tokens / (elapsed / 1000) if elapsed > 0 else 0 return GenerationState( status='complete', tokens_generated=total_tokens, tokens_per_second=round(tps, 1), elapsed_ms=int(elapsed), privacy_badge=True ) ``` The privacy badge isn't just visual—it creates user awareness of the tradeoff they're getting. Users who understand why local is different become advocates for your product. Model switching UX matters too. When users select between量化 models (7B, 13B, 70B), show the capability delta with real benchmarks from their hardware, not marketing specs.

EXERCISE

Build a UI component that displays generation progress without showing a traditional spinner. Use tokens-per-second as the primary feedback mechanism, and include a "processed locally" indicator that updates in real-time. Include memory usage to help users understand resource consumption.