Prompt
Definition pending
We've cataloged "Prompt" but haven't written a full definition yet. Definitions are hand-curated rather than auto-generated, so it takes time to cover every term.
Want this one prioritized? Email us and we'll bump it.
Practical example
A prompt is not just the user's question — it's the full input string: system prompt + conversation history + few-shot examples + RAG context + user query. Every token in the prompt costs VRAM and compute. A 4,000-token prompt on a 70B model consumes roughly 0.5 GB of KV cache. Chain-of-thought prompts can easily hit 8,000 tokens before the model generates a single word.
Workflow example
Audit your prompts: (1) log full prompt strings (strip PII first), (2) compute token count with the model's tokenizer (len(tokenizer.encode(prompt))), (3) if >50% of your prompt is system instructions + RAG context, optimize: compress system prompt, retrieve fewer but more relevant docs, (4) set max_prompt_tokens budget per endpoint: 2000 for latency-sensitive, 8000 for quality-sensitive. Monitor in Grafana.