What this does

Ollama supports OpenAI-compatible function calling for models like llama3.2, qwen2.5, and mistral. This guide enables tool use for local agents without cloud APIs.

Steps

Verify model supports function calling. Pull a compatible model.

ollama pull llama3.2

Call the Ollama API with tools. Use the OpenAI-compatible endpoint at /v1/chat/completions.

from openai import OpenAI

client = OpenAI(
    api_key="ollama",
    base_url="http://localhost:11434/v1"
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_time",
            "description": "Get current time for a timezone",
            "parameters": {
                "type": "object",
                "properties": {
                    "timezone": {"type": "string", "description": "Timezone like UTC, EST"}
                },
                "required": ["timezone"]
            }
        }
    }
]

response = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "What time is it in Tokyo?"}],
    tools=tools
)

print(response.choices[0].message.tool_calls)

Use LangChain with Ollama function calling. LangChain wraps the Ollama client.

from langchain_ollama import ChatOllama
from langchain.tools import tool

@tool
def get_time(timezone: str) -> str:
    """Get current time for a timezone."""
    import datetime
    return f"Current time in {timezone}: {datetime.datetime.now().isoformat()}"

llm = ChatOllama(model="llama3.2", temperature=0)
llm_with_tools = llm.bind_tools([get_time])

result = llm_with_tools.invoke("What time in EST?")
print(result.tool_calls)

Check if a model supports tools. Try with a simple prompt and inspect for tool_calls.

response = client.chat.completions.create(
    model="llama3.2",
    messages=[{"role": "user", "content": "Use get_time for UTC"}],
    tools=tools
)
has_tools = response.choices[0].finish_reason == "tool_calls"
print(f"Tool calling supported: {has_tools}")

Verification

curl -s http://localhost:11434/api/tags | python -c "import sys,json; data=json.load(sys.stdin); print([m['name'] for m in data['models']])"
# Expected: List of pulled models including llama3.2

Common failures

Model ignores tool calls. Not all models support function calling. Use llama3.2:3b, qwen2.5:7b, or mistral:7b.
Ollama version too old. Function calling requires Ollama 0.3+. Run ollama --version to check.
API key required but ignored. Ollama accepts any string for api_key, but it must be non-empty when using the OpenAI client.
Version mismatch - The installed package or runtime differs from the command shown; check the version first and rerun the smallest verification command.
Local environment drift - Another service, virtual environment, model, or path is being used; print the active binary path and configuration before changing the guide steps.

Related guides

How to Use OpenAI Function Calling with Tools
How to Define JSON Schema for Tool Calling