07. Router Chains

Router chains direct user queries to one of several specialized chains based on content analysis. Instead of an if/elif tree, you define candidate chains and a router that scores which one best handles the input. LangChain provides RouterChain (a LLMChain-powered router with configurable fallback) and MultiPromptChain (the declarative factory that builds the routing logic automatically).

The routing pattern has four components:

from langchain.chains import MultiPromptChain
from langchain.chains.router import LLMRouterChain, MultiPromptRouter
from langchain.prompts import PromptTemplate
from langchain_ollama import ChatOllama

llm = ChatOllama(model="llama3.2:3b", temperature=0)

# Define destination chain prompts
destination_chains = {
    "python": PromptTemplate.from_template(
        "You are a Python expert. Answer this Python question: {question}"
    ),
    "javascript": PromptTemplate.from_template(
        "You are a JavaScript expert. Answer this JavaScript question: {question}"
    ),
    "sql": PromptTemplate.from_template(
        "You are a SQL expert. Answer this database question: {question}"
    ),
}

# Create individual chains for each destination
chains = {
    name: LLMChain(prompt=prompt, llm=llm, verbose=False)
    for name, prompt in destination_chains.items()
}

# Build the routing prompt (tells the router how to classify)
router_template = PromptTemplate.from_template(
    """Given the user question, choose the best destination: {names}.
Question: {question}
Return ONLY the destination name, nothing else."""
)

router_chain = LLMRouterChain.from_llm(
    llm=llm,
    router_prompt=router_template,
)

# Wire it all together
chain = MultiPromptChain(
    router_chain=router_chain,
    destination_chains=chains,
    default_chain=LLMChain(
        prompt=PromptTemplate.from_template(
            "Answer this general question: {question}"
        ),
        llm=llm,
    ),
    verbose=True,
)

# Test routing
questions = [
    "How do I sort a list in Python?",
    "What's the difference between INNER JOIN and LEFT JOIN?",
    "Why is my website loading slowly?",
]

for q in questions:
    print(f"Q: {q}")
    result = chain.invoke({"question": q})
    print(f"A: {result['text']}\n")

The router chain responds with a destination name string and the MultiPromptChain dispatches to the corresponding entry in destination_chains. The default_chain handles cases where the router output doesn't match any destination name (empty output, tie, or unrecognized label).

Key detail: LLMRouterChain.from_llm() uses the raw LLM response as the routing key, not structured JSON or enum. This means whitespace, capitalization, and extra punctuation at the router's output can cause routing failures. The router prompt should instruct the model to output only the destination name with no punctuation.

For stronger typing, you can subclass RouterChain and implement Route logic manually:

from langchain.chains import RouterChain
from langchain.base import Chain

class IntentRouter(Chain):
    """Custom router using keyword matching—no LLM needed for simple cases."""
    
    def __init__(self, routes: dict[str, LLMChain]):
        self.routes = routes

    @property
    def input_keys(self) -> list[str]:
        return ["query"]

    @property
    def output_keys(self) -> list[str]:
        return ["response"]

    def _route(self, query: str) -> str:
        query_lower = query.lower()
        for keyword, destination in self.routes.items():
            if keyword in query_lower:
                return destination
        return list(self.routes.values())[0]  # fallback to first

    def invoke(self, inputs: dict) -> dict:
        query = inputs["query"]
        selected = self._route(query)
        response = selected.invoke({"question": query})
        return {"response": response["text"]}

# Usage
router = IntentRouter({
    "python": chains["python"],
    "javascript": chains["javascript"],
})

result = router.invoke({"query": "Explain decorators in Python."})

This keyword-based router has no LLM overhead and zero token cost, making it suitable for production systems with limited vocabulary domains.

Failure mode: the LLMRouterChain approach can misroute when the router model is smaller than the destination models—like routing with a 3B model to a 70B model. The 3B model may not have sufficient reasoning to correctly classify ambiguous queries. For high-stakes routing (financial, legal, medical domains), consider using a separate, stronger classification model or explicitly prompting the router with examples (few-shot routing).