13. Learning from Feedback

A personal AI agent that never adapts becomes stale. Users change their minds, requirements shift, and context evolves. OpenCLaw implements a feedback learning system that captures explicit corrections, implicit signals, and aggregated patterns to continuously improve response quality.

Feedback Capture Architecture

Feedback flows through a three-layer system. The capture layer intercepts user actions at interaction boundaries. The storage layer persists feedback with temporal context. The application layer updates model parameters and routing rules.

# feedback_capture.py
from dataclasses import dataclass
from datetime import datetime
from typing import Optional, Dict, Any
from enum import Enum

class FeedbackType(Enum):
    EXPLICIT_CORRECTION = "explicit_correction"
    IMPLICIT_ACCEPTANCE = "implicit_acceptance"
    REJECTION = "rejection"
    MODIFICATION = "modification"

@dataclass
class FeedbackEntry:
    interaction_id: str
    timestamp: datetime
    feedback_type: FeedbackType
    original_response: str
    user_modification: Optional[str] = None
    context_tags: Dict[str, Any] = None
    confidence_override: Optional[float] = None
    
    def validate(self) -> bool:
        if self.feedback_type == FeedbackType.EXPLICIT_CORRECTION:
            return self.user_modification is not None
        return True

class FeedbackCapture:
    def __init__(self, storage_backend):
        self.storage = storage_backend
        self.implicit_tracker = ImplicitSignalTracker()
    
    def record_explicit(self, entry: FeedbackEntry) -> str:
        entry.validate()
        return self.storage.insert(entry)
    
    def record_implicit(self, interaction_id: str, action: str) -> None:
        signal = self.implicit_tracker.analyze(action)
        if signal.strength > 0.7:
            entry = FeedbackEntry(
                interaction_id=interaction_id,
                timestamp=datetime.utcnow(),
                feedback_type=FeedbackType.IMPLICIT_ACCEPTANCE,
                original_response=signal.response,
                context_tags=signal.context
            )
            self.storage.insert(entry)

Implicit Signal Processing

Users reveal preferences through behavior rather than explicit statements. Copy-paste operations indicate partial satisfaction. Time spent reviewing suggests complexity concerns. Follow-up questions signal gaps.

The implicit tracker maintains a rolling window of user actions and correlates them with response characteristics. High correlation patterns become learning signals.

class ImplicitSignalTracker:
    def __init__(self, window_size: int = 50):
        self.window_size = window_size
        self.action_history = []
        self.response_mapping = {}
    
    def analyze(self, action: Dict[str, Any]) -> Signal:
        self.action_history.append(action)
        if len(self.action_history) > self.window_size:
            self.action_history.pop(0)
        
        response_fingerprint = action.get('response_hash')
        action_type = action.get('type')
        
        patterns = self.correlate_patterns(response_fingerprint, action_type)
        
        return Signal(
            strength=patterns.confidence,
            context=patterns.context,
            response=action.get('response')
        )
    
    def correlate_patterns(self, response_fp: str, action_type: str) -> Pattern:
        # Implementation details
        pass

Learning Rate Management

Feedback influence must be balanced to prevent overcorrection. A single outlier should not reshape behavior, but sustained patterns should. OpenCLaw uses exponential decay weighting combined with minimum sample thresholds.