16. Documentation

Chapter 16 of 18 · 20 min

KEY INSIGHT

Documentation is a user interface. Just as you would test user-facing interfaces, you should test documentation—through the eyes of someone who doesn't know your system. Documentation transforms code from "exists" to "usable." A well-documented system with mediocre code outlasts a poorly-documented system with excellent code. ### Documentation Tiers **API reference**: Complete, accurate descriptions of every public function, class, and module. Generated from docstrings using tools like Sphinx or pdoc. **Tutorial**: Step-by-step guide walking new users through a complete use case. Assumes minimal prior knowledge. Should be completable in 15-30 minutes. **How-to guides**: Solutions for specific common tasks. "How to evaluate on a custom dataset." "How to add a new model architecture." Assumes some familiarity. **Explanation**: Conceptual discussion of why the system works as it does. Architecture decisions, design trade-offs, background concepts. ```python # docstring_example.py def retrieve(query: str, top_k: int = 10, filters: dict = None) -> list[Document]: """Retrieve documents relevant to a query. Args: query: Natural language search query. top_k: Maximum number of documents to return. Default 10. filters: Optional metadata filters. Keys are field names, values are acceptable values. Default None. Returns: List of Document objects sorted by relevance score (descending). Empty list if no documents match. Raises: ValueError: If top_k < 1 or query is empty. ConnectionError: If the document store is unreachable. Example: >>> docs = retrieve("transformer architecture", top_k=5) >>> print(f"Found {len(docs)} documents") Found 5 documents """ ``` ### Writing for Users Technical documentation serves users with different goals: - **Evaluators**: Want to understand if the system meets their needs before committing time. Provide high-level overview, benchmark results, known limitations. - **Integrators**: Want to build on your system. Provide API reference, architecture overview, extension points. - **Debuggers**: Have a broken system. Provide troubleshooting guides, common error messages and solutions, logging configuration. ### Documentation Testing Documentation rots without maintenance. Test it: ```python # docs/test_docs.py import subprocess import pytest from pathlib import Path def test_readme_install(): """Verify README installation commands work.""" # This test validates that the documented install process works result = subprocess.run( ["pip", "install", "-e", "."], capture_output=True, timeout=300 ) assert result.returncode == 0, f"Installation failed: {result.stderr}" def test_examples_run(): """Verify all example scripts execute without error.""" examples_dir = Path("examples") for example in examples_dir.glob("*.py"): result = subprocess.run(["python", example], capture_output=True) assert result.returncode == 0, f"{example} failed: {result.stderr}" ``` ### Maintenance Practices - Treat documentation updates like code reviews—require PRs, review changes - Add "documentation needed" labels to issues when code lacks docs - Review docs during feature development, not after

EXERCISE

Write documentation for your research system covering: (1) a 3-minute quick start guide, (2) a description of a key design decision and its trade-offs, (3) a troubleshooting section for the top 3 failure modes you encountered during development. Test the quick start on a colleague with no prior exposure.