09. Packaging and Distribution

Packaging transforms your code into a product users can install and run. This chapter covers distribution formats for local AI products, considering the unique challenges of shipping products that include models, dependencies, and configuration.

Package Types

Choose package formats appropriate for your target users:

• Python packages via PyPI or conda for developer tools
• Executable binaries via PyInstaller, Nuitka, or Go/Rust compilation
• Container images via Docker for server deployment
• Application bundles for macOS (.app), Windows (.exe/.msi), or Linux (AppImage)

For most local AI products, Docker provides the best balance of portability and simplicity. Single-container images that include models, dependencies, and startup scripts work well for deployment scenarios.

# Dockerfile
FROM python:3.11-slim

WORKDIR /app

# Install system dependencies for model inference
RUN apt-get update && apt-get install -y \
    build-essential \
    && rm -rf /var/lib/apt/lists/*

# Copy requirements first for better caching
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

# Copy application code
COPY src/ ./src/
COPY models/ ./models/

# Set environment
ENV PYTHONUNBUFFERED=1

# Default command
CMD ["python", "-m", "src.api.routes"]

Model Distribution

AI models are large and often require special handling. Options include:

• Include models in the package (simple but large)
• Download models on first run (smaller package but requires internet)
• Reference external model sources (most flexible but requires configuration)

For MVP, include models in the package to ensure offline functionality. Optimize later if package size becomes problematic.

Configuration Management

Support multiple configuration methods in order of precedence: environment variables, config files, and command-line arguments. This allows users to customize behavior while providing sensible defaults.

# src/config/settings.py
import os
from functools import lru_cache

@lru_cache
def get_settings():
    return Settings(
        model_path=os.getenv("MODEL_PATH", "models/default"),
        max_results=int(os.getenv("MAX_RESULTS", "10")),
        log_level=os.getenv("LOG_LEVEL", "INFO"),
    )

class Settings:
    model_path: str
    max_results: int
    log_level: str

Versioning and Releases

Use semantic versioning (MAJOR.MINOR.PATCH) and tag releases in version control. Maintain a changelog documenting what changed in each release. Users need to understand the impact of updating.

Build Scripts

Create build scripts that automate packaging tasks. This ensures consistent builds and reduces manual errors.

#!/bin/bash
# scripts/build.sh
set -e

VERSION=${1:-dev}
BUILD_DIR="dist"

rm -rf $BUILD_DIR
mkdir -p $BUILD_DIR

echo "Building version $VERSION"

docker build -t my-product:$VERSION .
docker save my-product:$VERSION | gzip > $BUILD_DIR/my-product-$VERSION.tar.gz

echo "Build complete: $BUILD_DIR/my-product-$VERSION.tar.gz"