13. Remote Access Setup
Reliable remote access to an AI server requires SSH key authentication, session persistence, and optional tunneling for GPU monitoring tools.
SSH key authentication:
# On your local machine, generate a key with a comment
ssh-keygen -t ed25519 -f ~/.ssh/ai-server-key -C "ai-server-2026"
# Copy the public key
ssh-copy-id -i ~/.ssh/ai-server-key.pub user@your-server-ip
# Connect using the specific key
ssh -i ~/.ssh/ai-server-key user@your-server-ip
Persistent SSH sessions with tmux so inference continues even if your local terminal disconnects:
# Install tmux
sudo apt install tmux
# Create a named session
tmux new -s ai-work
# Inside tmux, run your commands
ollama serve
# Detach with Ctrl+B, then D
# Reattach later
tmux attach -t ai-work
SSH tunnel for accessing a web UI (like llama.cpp server's web interface) on a remote machine through your local browser:
# On your local machine
ssh -L 8080:localhost:8080 -i ~/.ssh/ai-server-key user@your-server-ip
# Now open http://localhost:8080 in your local browser
# All traffic is tunneled through SSH encrypted channel
GPU monitoring over tunnel:
# Create a persistent reverse tunnel
ssh -R 9000:localhost:9000 -N -f -i ~/.ssh/ai-server-key user@your-server-ip
# Prometheus at port 9000 on the server is now accessible at
# localhost:9000 on your local machine
Failure mode: SSH key still prompts for password. The ssh-copy-id command wrote to the wrong file or the PubkeyAuthentication directive in sshd_config is no. Check /etc/ssh/sshd_config and set PubkeyAuthentication yes, then sudo systemctl restart sshd.
Failure mode: tmux attach says no sessions. The session name was not saved. Always name sessions: tmux new -s persistent-name. Unnamed sessions are harder to find with tmux ls.
Failure mode: SSH tunnel works but the web service is not accessible. The remote service is bound to 127.0.0.1 (default for many services). Check netstat -tlnp | grep 8080 on the remote and look for 127.0.0.1:8080 vs 0.0.0.0:8080. Bind to 0.0.0.0 or use localhost in the tunnel URL.
Set up SSH key authentication, create a persistent tmux session, start Ollama inside it, detach, reconnect, and verify Ollama is still running. Create an SSH tunnel to access the Ollama API port locally.