spaCy

spaCy is a Python library for industrial-strength natural language processing (NLP) that provides pre-trained pipelines for tokenization, part-of-speech tagging, named entity recognition, dependency parsing, and text classification. Unlike general-purpose LLMs, spaCy is optimized for speed and low memory usage, running on CPU or GPU with minimal latency. Operators encounter spaCy when they need fast, deterministic text processing (e.g., extracting entities or lemmatizing text) before feeding data into a local LLM, or when building custom NLP pipelines that run on consumer hardware without requiring a GPU.

spaCy is designed for production use, offering pre-trained models for multiple languages that are small (typically 10-50 MB) and fast (processing thousands of documents per second on a CPU). Its pipeline components are modular: operators can add, remove, or replace steps like tokenizer, tagger, parser, and entity recognizer. spaCy also supports custom training using its spacy train command, allowing operators to fine-tune models on domain-specific data. Unlike Hugging Face Transformers, spaCy models are not transformer-based by default (though it supports transformer wrappers via spacy-transformers). For local AI workflows, spaCy is often used as a preprocessing step to clean or structure text before passing it to an LLM, reducing token count and improving output quality. It integrates with libraries like Ollama via custom components that call LLMs for entity resolution or text generation.

An operator building a local document classifier might use spaCy to extract named entities (e.g., dates, product names) from PDFs before feeding them into Llama 3.1 8B for summarization. With a 16 GB VRAM GPU, the spaCy pipeline runs on CPU in under 100 ms per document, leaving the GPU free for the LLM. The operator can run python -m spacy download en_core_web_sm to get a 12 MB model that handles tokenization and NER out of the box.

In a typical workflow, an operator runs spacy.load('en_core_web_sm') in a Python script to get a pipeline, then calls nlp(text) to process a string. The result is a Doc object with token-level attributes like .text, .pos_, .ent_type_. For integration with Ollama, the operator might write a custom spaCy component that sends extracted entities to an LLM via the Ollama API (ollama.chat(model='llama3.1', messages=[...])) and attaches the response as a custom extension.