15. Grafana Dashboards

Chapter 15 of 24 · 20 min

KEY INSIGHT

Dashboard design should prioritize the critical path: model availability, inference latency, and error rates appear first; secondary metrics appear in expandable sections. ### Dashboard Architecture Organize dashboards hierarchically from overview to detail: ``` Production Inference ├── Overview (system-wide health) ├── Model Performance │ ├── LLM Metrics │ ├── Embedding Metrics │ └── Classification Metrics ├── Infrastructure │ ├── GPU Utilization │ ├── Memory Usage │ └── Network I/O └── Cost Analysis ``` ### Template Variables Enable flexible dashboard filtering with template variables: ```bash # dashboard.json excerpt { "templating": { "list": [ { "name": "model", "type": "query", "query": "label_values(inference_requests_total, model_name)", "multi": true }, { "name": "instance", "type": "query", "query": "label_values(gpu_utilization, instance)", "multi": true } ] } } ``` ### Essential Panels **Inference Latency Distribution Panel:** ```json { "title": "Inference Latency P50/P95/P99", "type": "graph", "targets": [ { "expr": "histogram_quantile(0.50, rate(inference_latency_seconds_bucket{model=\"$model\"}[5m]))", "legendFormat": "P50" }, { "expr": "histogram_quantile(0.95, rate(inference_latency_seconds_bucket{model=\"$model\"}[5m]))", "legendFormat": "P95" }, { "expr": "histogram_quantile(0.99, rate(inference_latency_seconds_bucket{model=\"$model\"}[5m]))", "legendFormat": "P99" } ], "gridPos": {"h": 8, "w": 12, "x": 0, "y": 0} } ``` **GPU Utilization Heatmap:** ```json { "title": "GPU Utilization Heatmap", "type": "heatmap", "targets": [ { "expr": "rate(gpu_utilization_percent{instance=~\"$instance\"}[1m])", "bucketSet": { "colors": ["#00b894", "#fdcb6e", "#d63031"] } } ] } ``` ### Automated Alert Visualization Add status panels showing active alert counts: ```json { "title": "Alert Status", "type": "stat", "targets": [ { "expr": "_COUNT{alertname=~\".*\", status=\"firing\"}", "legendFormat": "Firing" }, { "expr": "COUNT(ALERTS{status=\"pending\"})", "legendFormat": "Pending" } ], "options": { "colorMode": "background", "colorValue": true } } ```

Visualization transforms raw metrics into actionable operational intelligence. Grafana dashboards serve two audiences: real-time operational monitoring during incidents, and historical analysis for capacity planning and optimization.

Local verification checkpoint

Run the smallest example from this chapter in a local workspace and record the package version, runtime, data path, and observed output. If the result depends on model size, vector count, CPU/GPU backend, or available memory, note that constraint beside the exercise so the lesson remains reproducible.

EXERCISE

Create a Grafana dashboard monitoring inference latency percentiles, request throughput, and GPU utilization for a local model serving deployment. Include template variables for filtering by model name and instance. Add threshold-based background coloring that highlights latency values exceeding SLO thresholds.