The sentence that cost 15,000 tokens

On a normal first turn, Qwen3.8-27B was averaging around 10,000 reasoning tokens before it produced any output at all. The worst turns hit 15,000. That's most of a 16,384-token budget spent thinking, on a GPU I rent by the hour.

I changed one string in the request. The peak dropped to 8,000, and the answers were exactly as good as before. Not a fine-tune, and not a different model. One sentence, which the model's own chat template had been adding on my behalf, and which I had never read.

Nymbus is my iOS app for talking to local models. When I want the 27B at real speed it points at a rented 5090 through a small proxy on my Mac, and the agent side of the app hands the model 13 tools.

For weeks the tool calls in long agent loops were erratic. A turn would run forever, hit the token cap, or run so long the phone gave up on the connection. Three lines from the proxy log, all from before the fix:

stream done: finish_reason=length              frames=32736  max_tokens=16384  tools=13
stream done: finish_reason=tool_calls          frames=28888  max_tokens=16384  tools=13
stream done: finish_reason=CLIENT-DISCONNECTED frames=18983  max_tokens=16384  tools=13

That first one burned the entire 16,384-token budget and got cut off mid-stream. Here's the part that hid the cause for so long: a truncated tool call and a malformed tool call look identical from the client side. Both land as broken JSON. So the whole thing read as a model that was bad at tool calls, and I spent my time staring at tool schemas.

llama-server has a POST /apply-template endpoint. It takes the same body as a chat completion and returns the fully rendered prompt string. No generation, no tokens spent, instant.

I'd never used it. Send it your real request, tools and all, and you get back the exact text the model sees once the chat template has done its work. It turned a week of guessing into one paste.

Here's what the template had been putting into every request, directly above the function definitions:

Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.

That's reasonable advice for a hard question with one right answer. Sitting on top of a roster of 13 tools, it's close to the worst thing you can say. "Consider plausible alternatives" is a standing instruction to walk the other twelve tools before committing to one, every single turn. Telling a model to weigh its alternatives right above a list of 13 tools is like handing somebody a menu and asking them to think hard about whether they'd rather eat somewhere else.

It also sits in a spot no ordinary system prompt occupies. The template renders it inside the tools block, above the function definitions, which is why a polite "be concise" in my own system prompt had never made a dent.

Both came out of reading the rendered prompt.

Both <code>reasoning_effort</code> at the top level and <code>chat_template_kwargs.reasoning_effort</code> reach the template. Either one works.

My first instinct was to reproduce it in isolation. I ran 100 calls across 5 conditions: plain tool prompts, deliberately vague ones like "the project seems disorganized, clean it up," and 5,000-token contexts.

Worst case across all 100 was 403 completion tokens. Nothing spiralled, in any condition.

That result was accurate and completely useless, and it took me longer than it should have to see why. A first turn has nothing to loop on. The model calls a tool and stops, because no result has come back yet for it to react to. The failure only exists once tool results are fed back in. You can't test agent behavior with single-shot calls, and starting with a real loop would have saved me a day.

So I rebuilt it as an actual agent loop. Same tasks, same synthetic tool results every run, 9 loops per arm.

                       xhigh (default)    low + custom
  median total                  2,593           1,128
  max total                     4,782           1,469
  worst single turn             2,889             833
  concluded on its own         1 of 9          4 of 9

Halving the token count is nice. Termination is the actual bug. At xhigh the model almost never decides it's finished. It keeps calling tools until a cap or a client timeout ends the turn on its behalf, which is precisely what those finish_reason=length lines in the log were.

One caveat, because I don't want anyone quoting the wrong figure. The synthetic tool results were trivial and identical on every run, which inflates the never-concludes rate in both arms equally. The comparison between the two columns holds. 1 in 9 is not a real-world termination rate.

A benchmark built on synthetic tool results tells you the direction. It doesn't tell you the size. So I ran the proxy with the injection on through an evening of ordinary Nymbus work and watched the reasoning counts.

First-turn thinking had been averaging around 10,000 tokens with peaks near 15,000. After the change, the peak was 8,000. That's a 47% cut in the tail, which is the only part that was doing damage. The tail is what hit the 16,384 cap and disconnected the phone at 19,000 frames.

Quality was unchanged. I went looking for a drop and couldn't find one, which makes sense once you see what those tokens were for. The model wasn't spending 15,000 tokens getting the answer right. It was spending them talking itself out of tools it was going to call anyway.

I spent weeks treating this as a model quality problem, then as a tool schema problem. It was a string. The model had been following an instruction handed to it by default, in the strongest position in the prompt, by a template I'd never opened.

When a local model behaves badly with tools, render the prompt before you change anything else. POST /apply-template costs nothing and shows you what you're really asking for. Half the time the answer is a sentence somebody else wrote.