02. Orchestrator-Worker Pattern

Chapter 2 of 24 · 15 min

KEY INSIGHT

Central orchestrators provide simplicity and controllability at the cost of scalability; use them when task graphs are known upfront and debuggability matters. The orchestrator-worker pattern places a central coordinator in control. The orchestrator receives incoming requests, decomposes them into subtasks, assigns each to appropriate workers, collects results, and assembles final responses. Workers remain stateless—they execute assigned tasks and return outputs without awareness of the broader pipeline. This pattern excels for well-structured workflows with known dependency graphs. Request processing pipelines, document analysis systems, and multi-step form handling map cleanly to orchestrated topologies. The central coordinator maintains visibility into pipeline state, enabling logging, error recovery, and retry logic in one place. Debugging simplifies when a single component drives execution. Tracing a request means following the orchestrator's logic, not reconstructing distributed causality. New developers understand system behavior by examining one class rather than mapping agent interactions. Failure handling proves straightforward. When a worker fails, the orchestrator detects timeout or error responses and can retry, route to a backup worker, or escalate to human review. The orchestrator owns retry budgets, backoff strategies, and circuit-breaker state. The pattern's weakness lies in centralization. The orchestrator becomes a bottleneck—every task passes through it, creating latency and limiting throughput. If the orchestrator crashes, the entire system halts. Scaling requires scaling one component that may not parallelize well. Task decomposition requires the orchestrator to understand enough about each subtask to assign correctly. In complex domains, building this knowledge proves difficult. The orchestrator may become a large, unwieldy prompt or a maze of conditional logic. ```python import asyncio from dataclasses import dataclass from typing import Any @dataclass class Task: id: str type: str payload: Any dependencies: list[str] class Orchestrator: def __init__(self, workers: dict[str, Any]): self.workers = workers self.results = {} async def execute(self, task: Task) -> Any: # Wait for dependencies for dep_id in task.dependencies: if dep_id not in self.results: raise ValueError(f"Missing dependency: {dep_id}") # Route to appropriate worker worker = self.workers.get(task.type) if not worker: raise ValueError(f"No worker for task type: {task.type}") result = await worker.process(task.payload) self.results[task.id] = result return result # Usage orchestrator = Orchestrator({ "analysis": AnalysisWorker(), "transformation": TransformWorker(), "formatting": FormatWorker() }) result = await orchestrator.execute( Task("t1", "analysis", {"text": raw_data}, []) ) ```

EXERCISE

Design an orchestrator for a content moderation system. Tasks include image analysis (classify inappropriate content), text analysis (detect policy violations), and notification dispatch (alert human reviewers). Specify the task graph, dependency relationships, and how the orchestrator handles cases where image and text analysis disagree. (12 minutes)