08. Integration Testing

Integration testing verifies that your components work together correctly. This chapter covers testing strategies for local AI products, where testing involves both deterministic logic and probabilistic model behavior.

Testing Strategy

Layer your tests from unit to integration. Unit tests verify individual functions work correctly in isolation. Integration tests verify components work together. End-to-end tests verify the complete user workflow.

For local AI products, the testing challenge is handling model outputs, which are inherently non-deterministic. You cannot test that a model returns a specific response, but you can test that responses meet criteria: output format, response length, processing time, or error handling.

# tests/test_integration.py
import pytest
from src.models.inference import LocalModel
from src.services.search import perform_search

@pytest.fixture
def model():
    return LocalModel("models/test-model")

@pytest.fixture
def indexed_docs(tmp_path):
    docs = tmp_path / "docs"
    docs.mkdir()
    (docs / "intro.txt").write_text("Introduction to the system")
    (docs / "guide.txt").write_text("User guide for configuration")
    index_documents(docs)
    return docs

def test_search_returns_formatted_results(indexed_docs):
    results = perform_search("configuration guide", max_results=5)
    
    assert len(results) <= 5
    assert all("path" in r for r in results)
    assert all("score" in r for r in results)

def test_search_handles_empty_query(indexed_docs):
    with pytest.raises(ValueError):
        perform_search("")

Test Fixtures and Data

Create realistic test fixtures that exercise edge cases. Include empty inputs, large inputs, malformed data, and boundary conditions. Your tests are only as good as the scenarios they cover.

For model testing, create a small test dataset with known expected behaviors. Use this for regression testing when updating models or inference logic.

Continuous Integration

Set up CI to run your test suite automatically on every change. This catches regressions before they reach users.

# .github/workflows/test.yml
name: Test Suite

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      - run: pip install -r requirements.txt
      - run: pytest tests/ --verbose

Performance Testing

Test that your product performs acceptably under load. Measure inference latency, indexing throughput, and memory usage. Define acceptable thresholds and fail builds that exceed them.

# tests/test_performance.py
import time
import pytest

def test_search_latency(indexed_docs):
    start = time.time()
    results = perform_search("configuration", max_results=10)
    elapsed = time.time() - start
    
    assert elapsed < 2.0, f"Search took {elapsed:.2f}s, expected < 2.0s"
    assert len(results) > 0

Test Reporting

Generate test reports that highlight failures, track coverage, and measure performance trends over time. This data informs optimization efforts and identifies fragile components.