How to build a decision tree prompt chain that routes user queries based on detected intent

What this does

A decision tree prompt chain classifies an incoming user query, determines the appropriate intent category, and routes the request down a specialized sub-chain tailored to that intent. This architecture improves accuracy over a single monolithic prompt because each branch can use deeply specialized instructions without interference from unrelated task demands.

Steps

1. Define the intent taxonomy and write a classification prompt that takes a raw user query and outputs one intent label from the taxonomy.
2. Register a function or tool definition that returns the intent label — this constrains the model to a valid choice.
3. Call the classifier with the user's query and retrieve the intent via structured output.
4. Build a routing map: intent_label -> prompt_template mapping for every intent in the taxonomy.
5. Retrieve the appropriate prompt template from the routing map based on the detected intent.
6. Inject the original user query into the selected prompt template, along with any session context.
7. Execute the specialized sub-chain prompt against the AI API.
8. Capture and return the sub-chain response to the user.
9. Add a fallback branch: if the classifier outputs an unknown intent or confidence is below a threshold, route to a generic handler.
10. Instrument the router with logging to capture intent counts, routing times, and fallback rates for analysis.

Verification

python3 -c "
import json, sys
from openai import OpenAI
client = OpenAI()

intent_tools = [{'type': 'function', 'name': 'route_intent', 'parameters': {'type': 'object', 'properties': {'intent': {'type': 'string', 'enum': ['technical_support', 'billing', 'feature_request']}}, 'required': ['intent']}}]

query = 'My invoice shows a charge I did not authorize.'
resp = client.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': f'Classify: {query}'}],
    tools=intent_tools,
    tool_choice={'type': 'function', 'function': {'name': 'route_intent'}}
)
tool_call = resp.choices[0].message.tool_calls[0]
print(json.loads(tool_call.function.arguments)['intent'])
"

Expected output: billing.

Common failures

• Classifier outputs intent outside taxonomy: model produces an unsupported label. Solution: add a validation layer after the call that re-maps unknown outputs to the closest taxonomy entry or triggers the fallback.
• Ambiguous queries produce low-confidence routing: borderline cases route incorrectly. Solution: add a confidence threshold check — queries below threshold receive a disambiguation sub-prompt before routing.
• Sub-chain prompt drift causes loop cycles: certain intents trigger routing back to the classifier. Solution: add a depth counter that aborts and returns a graceful error after a configurable number of hops.
• Intent taxonomy too coarse: broad categories cause sub-chain prompts to produce generic, unhelpful responses. Solution: refine taxonomy iteratively using cluster analysis of actual query embeddings.

Related guides

• chain-prompts-complex-tasks-sequential
• setup-ab-testing-prompts