Rotary Position Embedding (RoPE)
Definition pending
We've cataloged "Rotary Position Embedding (RoPE)" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
RoPE (Rotary Position Embedding) encodes position by rotating token vectors. It's what modern models (Llama, Mistral, DeepSeek) use for positional encoding. Key property: it naturally handles relative positions, and you can rescale it to extend context length without retraining. The "rope_theta" value in model configs controls the base frequency — higher = better at long-range positions.
Workflow example
RoPE configuration in llama.cpp: --rope-freq-base 1000000 --rope-freq-scale 0.5. If extending context from 8K to 32K, set rope-freq-base to ~1,000,000 (the LLaMA community standard). Test with "needle in a haystack" — place a fact at position 30,000 and ask the model about it. If it fails, increase rope-freq-base. If perplexity on short documents degrades, decrease it. This is a direct quality/length trade-off.