llama.cpp b9982: Per-Request Reasoning Budget Tokens Now Respected
▼ WHAT HAPPENED
In the latest release of llama.cpp (b9982), a critical bug has been fixed where per-request reasoning budget tokens were being ignored in chat completions. Previously, only server-level defaults and Anthropic-style aliases were considered, leading to potential misconfiguration or unintended behavior when attempting to override these settings on a per-request basis. This update ensures that any specified reasoning_budget_tokens value is now correctly applied, allowing for more granular control over the model's response generation process. Operators running local AI services should be aware of this change as it directly impacts how they manage and configure their chat completion requests.
▼ OPERATOR ANGLE
Review your API request configurations to ensure proper use of reasoning_budget_tokens for per-request overrides.