12. Firewall and Security
AI inference servers expose network ports that must be protected. On a local network this matters less; on a public or shared network it matters significantly.
Check current firewall status:
sudo ufw status verbose
# Status: active
# Logging: low
# Default: deny incoming (on Any), allow outgoing (on Any)
Basic secure configuration:
sudo ufw default deny incoming
sudo ufw allow ssh
sudo ufw allow 443/tcp # HTTPS for API access
sudo ufw allow 8080/tcp # Only if explicitly needed, restrict to IP range
sudo ufw enable
sudo ufw status numbered
Rate limiting at the firewall prevents abuse:
sudo ufw limit 22/tcp
# Inserts a rule that drops connections if more than 5 connections
# per 30 seconds are initiated from a single IP
For the Ollama API on port 11434, restrict to your local network:
sudo ufw allow from 192.168.1.0/24 to any port 11434
Docker and UFW interaction: Docker modifies iptables directly when you publish ports with -p 80:80. UFW rules may not apply to container-published ports. Verify with:
sudo iptables -L -n | grep 11434
If the rule is not there, the container published a port that bypasses UFW. Either publish only on localhost (-p 127.0.0.1:11434:11434) or add a Docker UFW after.rules entry:
sudo nano /etc/ufw/after.rules
# Add before the final COMMIT:
# Block external access to Ollama
-A INPUT -p tcp --dport 11434 -j DROP
Fail2ban for brute force protection:
sudo apt install fail2ban
sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local
[sshd]
enabled = true
port = ssh
maxretry = 3
bantime = 3600
findtime = 600
sudo systemctl enable fail2ban
sudo systemctl start fail2ban
sudo fail2ban-client status
Failure mode: UFW is inactive and Docker ports are exposed publicly. nmap -p 11434 your-server-ip returns open. Fix: sudo ufw enable and publish containers on 127.0.0.1 only.
Failure mode: SSH access is blocked after enabling UFW because SSH was not allowed first. Always run sudo ufw allow ssh before sudo ufw enable. If locked out, access via console or IPMI/iDRAC/IDRAC and fix the rule.
Failure mode: Fail2ban bans the wrong IP (your own IP) because you accessed the server from a NAT gateway. Add your IP to ignoreip in jail.local: ignoreip = 127.0.0.1/8 192.168.1.0/24.
Configure UFW to deny all incoming except SSH and the AI API port, install and configure fail2ban, publish the Ollama API on 127.0.0.1 only, and verify external port scans show only allowed ports.