How to design effective system prompts that guide an AI assistant behavior and constraints

What this does

A system prompt establishes the persistent behavior, output format, and constraints for an AI assistant across all interactions. This guide covers how to write, structure, and refine a system prompt that produces consistent, high-quality responses tailored to a specific workflow or team.

Steps

1. Define the core purpose of the assistant in a single clear sentence at the top of the system prompt (the role statement).
2. Enumerate output format requirements explicitly: preferred language, structure (bullet points vs. paragraphs), and any required sections or headers.
3. Specify behavioral constraints: topics the assistant must decline, actions it must avoid, and guardrails for sensitive operations.
4. Add instructions for how the assistant should handle ambiguity, such as asking clarifying questions instead of guessing.
5. Include a short example of an ideal response formatted exactly as expected (a few-shot instruction block).
6. Set the tone and communication style, for example: concise, technical, step-by-step, or formal.
7. Test the prompt by sending five diverse queries and evaluating the consistency of responses.
8. Refine the prompt based on observed deviations, adding clarifications or additional constraints as needed.

Verification

cat "Write a short HTTP GET function" | ollama run mistral --system "You are a Python expert. Always include type hints and docstrings in your responses."

Expected output (first line check):

def get_request(url: str) -> requests.Response:
    """Perform an HTTP GET request to the given URL."""

The output must include type hints and a docstring, confirming the system prompt is respected.

Common failures

1. System prompt is ignored on some queries — The prompt may be too long or include conflicting instructions. Simplify to a single-purpose directive per prompt and use separate sessions for different roles.
2. Output format is inconsistent across queries — The format instructions may not be explicit enough. Replace vague guidance like "be concise" with specific requirements such as "respond in three bullet points."
3. The assistant provides harmful or out-of-scope responses — Behavioral constraints are missing. Add a clear refusal instruction: "If a request violates safety or policy, respond with: I cannot help with that."
4. The few-shot example is not followed — The example may not match the query style. Ensure the example is short, uses the same format as the desired output, and appears at the end of the system prompt.

Related guides

• Create multi-role system prompts with distinct personas (guide 31)
• Test and iterate on system prompt designs (guide 32)