Transformer
Definition pending
We've cataloged "Transformer" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
The transformer architecture (2017) is the foundation of ALL modern LLMs. The key innovation: self-attention — each token looks at every other token, enabling parallel processing and capturing long-range dependencies. This replaced RNNs for NLP and is expanding to vision, audio, and beyond. It's the single most important architecture in AI today.
Workflow example
Transformer deployment: (1) every major LLM (GPT, Llama, Claude, Mistral) is a transformer, (2) transformer variants: decoder-only (GPT, Llama) for generation, encoder-only (BERT) for understanding, encoder-decoder (T5) for translation, (3) the attention mechanism's O(n²) complexity is the challenge — context length 2× = compute 4×, (4) for production: you never build a transformer from scratch — you fine-tune pre-trained ones, (5) hardware: transformers NEED GPU/accelerator — the matrix multiplications in attention and FFN are GPU-native operations.