HOW-TO · RAG

How to Integrate Web Search Tool in Agents

intermediate20 minBy Fredoline Eruo
Target environment
Ubuntu 24.04 · Ollama 0.4.x
PREREQUISITES

Agent with tool use, search API or local search engine, Python 3.10+

What this does

A web search tool lets agents retrieve real-time information from the internet — news, documentation, pricing — expanding their knowledge beyond what the training data covers.

Steps

  • Create a search function using Tavily. Tavily is optimized for LLM search.
import os
from tavily import TavilyClient

tavily_client = TavilyClient(api_key=os.environ["TAVILY_API_KEY"])

def search_web(query: str, max_results: int = 5) -> str:
    """Search the web and return results."""
    response = tavily_client.search(query=query, max_results=max_results)
    results = []
    for r in response["results"]:
        results.append(f"- [{r['title']}]({r['url']}): {r['content'][:200]}")
    return "\n".join(results)
  • Register the search as a LangChain tool.
from langchain.tools import tool

@tool
def web_search(query: str) -> str:
    """Search the web for current information on any topic."""
    return search_web(query)
  • Integrate into the agent loop. The agent decides when to search.
tools = [web_search]
llm_with_tools = llm.bind_tools([{
    "type": "function",
    "function": {
        "name": "web_search",
        "description": "Search the web for current information",
        "parameters": {
            "type": "object",
            "properties": {
                "query": {"type": "string", "description": "Search query"}
            },
            "required": ["query"]
        }
    }
}])
  • Add fallback for when API is unavailable. Use a local DuckDuckGo search.
from duckduckgo_search import DDGS

def search_fallback(query: str) -> str:
    try:
        with DDGS() as ddgs:
            results = list(ddgs.text(query, max_results=5))
            return "\n".join(f"- {r['title']}: {r['body'][:200]}" for r in results)
    except Exception as e:
        return f"Search unavailable: {e}"

@tool
def web_search(query: str) -> str:
    """Search the web. Uses primary API with fallback."""
    try:
        return search_web(query)
    except Exception:
        return search_fallback(query)
  • Cache search results to avoid duplicate API calls.
from functools import lru_cache

@lru_cache(maxsize=100)
def cached_search(query: str) -> str:
    return search_web(query)

Verification

python -c "
# Test DuckDuckGo fallback
from duckduckgo_search import DDGS
with DDGS() as ddgs:
    r = list(ddgs.text('python programming', max_results=1))
    print(len(r))
# Expected: 1
"

Common failures

  • API rate limits. Search APIs enforce limits (e.g., 100 req/day on free tier). Implement caching and rate limiting.
  • Irrelevant results. Short queries return noisy results. Have the agent generate detailed, specific queries.
  • Search timeout. The API may take 5-10 seconds. Set a reasonable timeout (10s) and handle Timeout exceptions gracefully.
  • Version mismatch - The installed package or runtime differs from the command shown; check the version first and rerun the smallest verification command.
  • Local environment drift - Another service, virtual environment, model, or path is being used; print the active binary path and configuration before changing the guide steps.

Related guides

  • How to Build Web Search with Citation Support
  • How to Add Rate Limiting to Agent Tool Calls