Installing Your First Local AI — What is Local AI — And Why It Matters (Chapter 8)

Choosing Your Tool

Before you install anything, understand the landscape:

Ollama: The simplest option. Single command installation. Works on Mac, Linux, and Windows. Good for beginners. Excellent model management.

LM Studio: Desktop application with a GUI. Easy to use. Good model management. Runs on Mac, Linux, Windows.

Jan: Open-source, privacy-focused. Desktop app. Runs fully offline. Good option if privacy is a priority.

llama.cpp: Command-line tool, more technical. The underlying engine that others build on. High learning curve but maximum control.

For your first installation, we recommend Ollama—it's the lowest barrier to entry while being production-quality.

Installing Ollama

macOS

# Download from https://ollama.ai/download
# Or use Homebrew:
brew install ollama

After installation, Ollama runs as a background service. You interact with it via command line.

Windows

Go to https://ollama.ai/download
Download the Windows installer
Run the installer
Ollama installs to your start menu

That's it. No complex configuration.

Linux

curl -fsSL https://ollama.ai/install.sh | sh

Your First Model

With Ollama installed, let's get a model running.

Step 1: Pull a model

# Pull the smallest practical model: TinyLlama (1.1B params)
ollama pull tinyllama

# This downloads the model file. For TinyLlama, it's ~600MB.

Step 2: Run the model

ollama run tinyllama

You'll see a prompt:

>>> Send a message (/? for help)

Type something:

>>> What is the capital of France?
Paris is the capital of France. It is also the largest city in the country 
and one of the most populous in Europe.

You just ran your first local AI model.

Step 3: Try a better model

# Pull a more capable model (Llama 3.2 7B, ~4GB)
ollama pull llama3.2:7b

# Run it
ollama run llama3.2:7b

You can switch between models:

ollama list  # shows installed models
ollama run llama3.2:7b  # runs specific model

Common Installation Issues

"Ollama command not found" on macOS/Linux:

# If you installed via script, verify the PATH
echo $PATH | grep ollama

# If not in path, add it
export PATH="$PATH:/usr/local/bin"

Out of disk space: Ollama downloads models to ~/.ollama/models. You can move this directory:

# Set OLLAMA_MODELS to a different path before installing
export OLLAMA_MODELS=/path/to/larger/disk/models

Slow download: Model files are large (4-8GB for good models). Use a fast internet connection. If it fails, ollama pull resumes automatically.