07. Network Security

Network security for local AI controls which systems can reach your AI services and what destinations those services can reach. Default-deny is the operating principle: block everything, then explicitly allow necessary communication.

Network topology for local AI:

Internet
    |
[Router/Firewall]
    |
[DMZ] - Web servers, if any
    |
[Internal Network] - Workstations, servers
    |
[AI Subnet] - GPU servers, Ollama hosts
    |
[Data Subnet] - Vector DB, document storage

Services on the AI subnet should not initiate connections to the internet (preventing data exfiltration) but may need to reach internal data stores. The DMZ has no direct path to the data subnet.

Default deny firewall rules:

# Flush existing rules and set default deny
iptables -F
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT DROP

# Allow localhost
iptables -A INPUT -i lo -j ACCEPT
iptables -A OUTPUT -o lo -j ACCEPT

# Allow established connections
iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow AI services to reach internal data stores
iptables -A OUTPUT -d 10.0.2.0/24 -p tcp --dport 5432 -j ACCEPT
iptables -A OUTPUT -d 10.0.2.0/24 -p tcp --dport 6379 -j ACCEPT

# Allow management access from admin subnet
iptables -A INPUT -s 10.0.0.0/24 -p tcp --dport 22 -j ACCEPT

# Log dropped packets (for forensics)
iptables -A INPUT -m limit --limit 5/min -j LOG --log-prefix "IPT_INPUT_DROP: "
iptables -A OUTPUT -m limit --limit 5/min -j LOG --log-prefix "IPT_OUTPUT_DROP: "

Monitoring network activity:

# Monitor new connections to AI services
sudo tcpdump -i eth0 -n 'tcp and port 11434 and tcp[tcpflags] == tcp-syn' \
    | while read line; do
        echo "$(date): $line"
    done > /var/log/ai-connections.log

# Alert on outbound connections from AI subnet (potential exfiltration)
iptables -A OUTPUT -s 10.0.1.0/24 ! -d 10.0.0.0/8 -m limit --limit 1/min \
    -j LOG --log-prefix "AI_EXFIL_ALERT: "