COURSE · FND · B003
Ollama — Installation to Mastery
Learn ollama — installation to mastery through RunLocalAI's practical lens: ollama, setup, models and cli, hardware fit, runtime settings, verification habits and local-vs-cloud tradeoffs.
PREREQUISITES
- B001
Course B003: Ollama — Installation to Mastery
Why this course exists
Most AI tutorials skip the boring parts: what happens when your model fails to download, your GPU sits idle, or your API returns a 404 because you forgot to start the server. This course covers the full lifecycle of running Ollama-from first install to production-ready deployments with monitoring, automation, and troubleshooting. You will encounter real failure modes and learn how to diagnose them methodically. By the end, you will know how to keep Ollama running reliably across different operating systems and hardware configurations.
What you will know after
- Install and verify Ollama on Linux, macOS, and Windows with GPU support configured
- Create and deploy custom model configurations using Modelfiles with proper parameterization
- Build applications that interface with Ollama via REST API and Python client with proper error handling
- Deploy Ollama in Docker containers with proper volume management and GPU passthrough
- Diagnose and resolve the five most common Ollama failures: GPU detection, OOM errors, slow inference, model download failures, and API connection issues
CHAPTERS
- 01What is Ollama?Ollama abstracts away the complexity of model serving while exposing low-level controls through Modelfiles and environment variables.15 min
- 02Installation by OSOllama installs differently on each OS but exposes the same API on port 11434-know your install location and service management approach.20 min
- 03First ModelRunning a model interactively and via API confirms the installation works. Save the timing metrics-they become baseline numbers for performance tuning later.20 min
- 04Multiple ModelsMultiple models consume independent resources. Monitor with `ollama ps` and stop unused models before loading new ones to avoid memory contention.20 min
- 05Modelfiles: Customizing ModelsModelfiles let you create reproducible model configurations with baked-in behavior. The base model is not copied-only the instructions are stored, so changes to the Modelfile affect subsequent runs.20 min
- 06Ollama REST APIThe API is stateless per request (except `/api/chat` which requires conversation history in the payload). For persistent sessions, implement client-side history management.20 min
- 07Ollama Python ClientThe Python client wraps the REST API with typed responses. It handles connection errors and provides streaming as generators for real-time output.20 min
- 08GPU vs CPU InferenceOllama auto-detects GPUs but may fall back to CPU if drivers are missing or memory is insufficient. Check `ollama ps` after loading a model to verify which processor is active.20 min
- 09Performance TuningThe fastest model is useless if the output quality suffers. Tune `temperature` and `num_ctx` for your use case, then measure actual throughput with `eval_count` and `eval_duration`.20 min
- 10Concurrent RequestsOllama queues requests by default. Increase parallelism only if you have sufficient GPU memory-otherwise, you trade throughput for latency without actual improvement.20 min
- 11Model Management AutomationAutomation scripts handle the non-interactive parts of model management. Build them with retry logic and logging from the start-debugging a failed deployment at 2 AM is harder than writing scripts with error handling now.20 min
- 12Ollama as Systemd ServiceSystemd manages the lifecycle of the Ollama process. Use override files to change environment variables without modifying the installed service file-updates to Ollama will overwrite the original.20 min
- 13Docker DeploymentDocker volumes persist models between container lifecycles. Without a volume, every `docker run` starts with no models installed.20 min
- 14Docker Compose StackDocker Compose lets you define the entire stack in a single file. Use host-mounted volumes for model directories when you need to access files from the host, and named volumes for portability.20 min
- 15Open WebUI IntegrationOpen WebUI runs separately from Ollama and communicates via HTTP. The connection URL must be reachable from the container-use Docker networking or host.docker.internal for host connections.20 min
- 16Continue.dev IntegrationContinue uses Ollama for both chat and autocomplete, but autocomplete requires fast response times. Use smaller models for autocomplete and reserve larger models for complex code questions in chat.15 min
- 17GPU Not DetectedGPU detection failures usually stem from missing drivers or missing container runtimes. `nvidia-smi` must work on the host before Ollama can use the GPU.20 min
- 18OOM ErrorsOOM errors are common with large models on limited hardware. Start with smaller models, monitor memory usage, and reduce context window before trying larger models.20 min
- 19Slow Inference DebuggingSlow inference is usually caused by one bottleneck: CPU-only mode, low GPU utilization, or thermal throttling. Measure systematically to identify which.20 min
- 20Model Download FailuresDownload failures are usually network-related. Check connectivity first, then disk space, then clear corrupted caches. As a last resort, import models manually from GGUF files.20 min