20. Ansible for AI

Chapter 20 of 24 · 30 min

KEY INSIGHT

Ansible provides idempotent, repeatable configuration management for AI infrastructure. Its agentless architecture and declarative language make it suitable for managing ML serving environments across diverse hardware without requiring installed agents on target systems. ### Why Ansible for Local AI Local AI deployments often span heterogeneous environments—different hardware generations, OS versions, and runtime configurations. Ansible's push-based model connects from a control node to targets via SSH, requiring no software installation on managed hosts beyond an SSH server. Ansible tasks are idempotent: running the same task multiple times produces the same result as running it once. This property is essential for reliable infrastructure management where repeated execution is common. ### Inventory and Pattern-Based Management ```yaml # YAML: Ansible inventory for local AI infrastructure # File: inventory.ini [ml_serving] # Production inference servers prod-inference-01 ansible_host=10.0.1.10 ansible_user=mlops prod-inference-02 ansible_host=10.0.1.11 ansible_user=mlops prod-inference-03 ansible_host=10.0.1.12 ansible_user=mlops [ml_serving:vars] ansible_python_interpreter=/usr/bin/python3 model_registry_base=s3://models/internal/ monitoring_enabled=true [ml_training] # Training and experiment servers train-01 ansible_host=10.0.2.10 ansible_user=mlops train-02 ansible_host=10.0.2.11 ansible_user=mlops [edge_nodes] # Edge deployment nodes with limited resources edge-retail-01 ansible_host=192.168.1.101 ansible_user=pi edge-retail-02 ansible_host=192.168.1.102 ansible_user=pi [edge_nodes:vars] ansible_python_interpreter=/usr/bin/python3 # ARM-specific Python path model_registry_base=s3://models/internal-arm/ [ml_serving:children] ml_training # Training servers inherit serving group vars ``` ### ML-Specific Ansible Roles ```yaml # YAML: Ansible role for ML runtime installation # File: roles/ml-runtime/tasks/main.yml - name: Check for required architecture assert: that: - ansible_architecture in ['x86_64', 'aarch64', 'armv7l'] fail_msg: "Unsupported architecture: {{ ansible_architecture }}" - name: Install system dependencies (Debian/Ubuntu) apt: name: - python3 - python3-pip - python3-venv - git - wget - curl state: present update_cache: yes when: ansible_os_family == "Debian" - name: Install system dependencies (RHEL/CentOS) dnf: name: - python3 - python3-pip - git - wget - curl state: present when: ansible_os_family == "RedHat" - name: Create ML service user user: name: ml-service shell: /bin/false home: /opt/ml-service system: yes create_home: yes - name: Install CUDA Toolkit (GPU nodes only) block: - name: Add NVIDIA repository get_url: url: https://developer.download.nvidia.com/compute/cuda/repos/{{ ansible_distribution_file_variance | lower }}/{{ ansible_architecture }}/cuda-keyring_1.0-1_all.deb dest: /tmp/cuda-keyring.deb - name: Install CUDA keyring apt: deb: /tmp/cuda-keyring.deb - name: Install CUDA runtime apt: name: - cuda-runtime-12-1 - libcublas-12-1 state: present when: (ansible_local | default({})).cuda | default(false) ``` ### Model Deployment Playbook ```yaml # YAML: Complete model deployment playbook # File: deploy-ml-models.yml --- - name: Deploy ML Models to Production hosts: ml_serving gather_facts: yes become: yes vars: model_artifact_path: /opt/models serving_port: 8080 health_check_path: /health deployment_warmup_seconds: 30 pre_tasks: - name: Verify model versions are specified assert: that: - model_versions is defined - model_versions | length > 0 fail_msg: "model_versions variable must be defined" - name: Check connectivity to model registry uri: url: "https://models.internal/v1/health" method: GET status_code: 200 register: registry_check failed_when: false - name: Ensure deployment is approved assert: that: deployment_approved | default(false) | bool fail_msg: "deployment_approved must be set to true" when: enforcement_mode == "strict" tasks: - name: Create model deployment directory file: path: """{{ model_artifact_path }}/{{ item.key }}""" state: directory owner: ml-service group: ml-service mode: '0750' loop: """{{ model_versions | dict2items }}""" - name: Download model artifacts aws_s3: bucket: mlops-models object: """{{ item.value.model_path }}""" dest: """{{ model_artifact_path }}/{{ item.key }}/model.tar.gz""" mode: get overwrite: different loop: """{{ model_versions | dict2items }}""" register: download_results - name: Extract model artifacts unarchive: src: """{{ model_artifact_path }}/{{ item }}/model.tar.gz""" dest: """{{ model_artifact_path }}/{{ item }}/""" remote_src: yes creates: """{{ model_artifact_path }}/{{ item }}/model.pkl""" loop: """{{ model_versions.keys() | list }}""" - name: Generate serving configuration template: src: model_config.yaml.j2 dest: """{{ model_artifact_path }}/{{ item.key }}/config.yaml""" loop: """{{ model_versions.keys() | list }}""" notify: restart model service - name: Verify model loads correctly shell: | cd """{{ model_artifact_path }}/{{ model_name }}""" python3 -c "import joblib; joblib.load('model.pkl')" args: executable: /bin/bash loop: """{{ model_versions.keys() | list }}""" vars: model_name: """{{ item }}""" register: model_load_check failed_when: false - name: Assert model loading succeeded assert: that: item.rc == 0 fail_msg: """Model {{ item.invocation.module_call_args[0].loop_var }} failed to load""" loop: """{{ model_load_check.results }}""" when: model_load_check.results | length > 0 - name: Enable and start model services systemd: name: """ml-model-{{ item.key }}.service""" state: started enabled: yes loop: """{{ model_versions | dict2items }}""" - name: Wait for model endpoints to become healthy uri: url: "http://localhost:"""{{ serving_port }}/{{ health_check_path }}""" method: GET status_code: 200 register: health_check until: health_check.status == 200 retries: 10 delay: 5 loop: """{{ model_versions.keys() | list }}""" vars: serving_port: """{{ 8080 + (loop_index | int) }}""" - name: Record deployment postgresql_stmt: query: | INSERT INTO deployment_log (model_name, version, host, deployed_at, deployed_by) VALUES (%s, %s, %s, NOW(), %s) login_host: postgres.internal login_db: mlops login_user: ansible login_password: """{{ lookup('env', 'DB_PASSWORD') }}""" loop: """{{ model_versions | dict2items }}""" handlers: - name: restart model service systemd: name: """ml-model-{{ inventory_hostname }}.service""" state: restarted - name: reload systemd systemd: daemon_reload: yes post_tasks: - name: Verify all endpoints responding uri: url: "http://{{ ansible_host }}:8080/v1/models" method: GET status_code: 200 register: endpoint_verify - name: Display deployment summary debug: msg: | Deployment complete for {{ inventory_hostname }} Models: {{ model_versions.keys() | list }} Results: {{ download_results.results | map(attribute='dest') | list }} ``` ### Testing Ansible Deployments ```yaml # YAML: Ansible deployment testing with molecule # File: molecule/default/molecule.yml --- dependency: name: galaxy driver: name: docker platforms: - name: instance image: docker.io/pyiron/centos7-systemd-python pre_build_image: yes command: /usr/sbin/init privileged: true provisioner: name: ansible verifier: name: testinfra additional_files: - test_deploy.py scenario: name: default test_sequence: - destroy - create - prepare - converge - verify # Run test cases - destroy ``` ---

EXERCISE

Define an Ansible inventory for your local AI deployment targets. Create a role for installing model serving dependencies. Write a deployment playbook that downloads artifacts, extracts them, configures serving, and verifies health. Run the playbook against a test environment and verify successful deployment.