12. Cloud-Fallback Strategy

Cloud-first strategy assumes external models provide the best capability-to-cost ratio for most workloads. Local resources activate only when cloud services become unavailable or economically inefficient. This approach minimizes local infrastructure investment and provides access to frontier model capabilities without hardware procurement.

The cloud-fallback implementation prioritizes geographic distribution and provider redundancy. Multi-region cloud endpoints reduce latency for distributed user bases. Provider redundancy—maintaining credentials for two or more cloud services—protects against single-provider outages affecting availability.

class CloudFirstRouter:
    def __init__(self, primary: CloudProvider, 
                 secondary: CloudProvider,
                 local: LocalModel):
        self.providers = [primary, secondary]
        self.local = local
        self.health_monitor = HealthMonitor()
    
    async def route(self, request: Request) -> Response:
        for provider in self.providers:
            if self.health_monitor.is_healthy(provider):
                try:
                    return await provider.inference(request)
                except ProviderError as e:
                    self.health_monitor.mark_failure(provider, e)
                    continue
        
        # All cloud providers failed; attempt local
        if self.local.can_handle(request):
            return await self.local.inference(request)
        
        # Local also unavailable; return queued response or error
        return await self._queue_request(request)

Cloud-first introduces dependencies on external service reliability. thorough monitoring must track not just request success rates but provider latency trends, error rate patterns, and capacity headroom. Proactive failover—moving traffic before services fail completely—requires sophisticated health assessment beyond simple availability checks.

Cost management becomes critical in cloud-first deployments. Without careful controls, cloud costs scale directly with usage, creating budget unpredictability. Rate limiting, request batching, and model-downgrade paths provide cost stability while maintaining capability.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.