Encoder-Decoder Transformer

Encoder-decoder transformers (T5, BART, original "Attention is All You Need" architecture) have two halves: an encoder reads the input bidirectionally, a decoder generates output autoregressively while cross-attending to encoder outputs.

Strengths: well-suited to translation, summarization, and structured input→output tasks. The encoder can use bidirectional attention, giving it stronger representation of the input than a decoder-only model can.

Modern open-weight LLMs are mostly decoder-only because scaling laws favored the simpler architecture and the gap closed with larger context. Encoder-decoder remains relevant in specialty translation and embedding models (some ColBERT variants, multilingual T5).

A team building a local translation service initially reaches for a decoder-only 7B chat model prompted with "translate this to French," but switches to a fine-tuned NLLB (No Language Left Behind) variant — an encoder-decoder model — once quality on low-resource language pairs becomes the bottleneck. The encoder reads the full source sentence bidirectionally before the decoder generates a single token, giving it a fundamentally better grip on source-side context than a causal-only model attending left-to-right. The tradeoff shows up in deployment: encoder-decoder models don't fit the same continuous-batching, KV-cache-reuse patterns that llama.cpp and vLLM optimize for decoder-only LLMs, so the team ends up running it through a dedicated translation-serving stack (CTranslate2) rather than their existing local-LLM inference server.