Two and a half minutes of silence

Two minutes and thirty-seven seconds. That's what 15,000 reasoning tokens costs at the ~95 tokens per second this model actually runs at, and 15,000 was the peak my first turns were hitting before Qwen3.8-27B lifted a finger. A typical turn ran 10,000 tokens, which is 1m 45s of nothing on screen. On a GPU I rent by the hour.

I changed one string in the request. The peak came down to 8,000 tokens, 1m 24s, and the answers were exactly as good as they'd been. Not a fine-tune, and not a different model. One sentence, which the model's own chat template had been adding on my behalf, and which I had never read.

Nymbus is my iOS app for talking to local models. When I want the 27B at real speed it points at a rented 5090 through a proxy on my Mac, and the agent side hands the model 13 tools.

The model is good. It picks the right tool, fills the arguments correctly, and the work it comes back with is work I keep. What it wouldn't do is get moving. Every turn opened with a long stretch of deliberation before the first action, and in an agent loop that tax gets paid again every turn.

Some turns never reached the end. Three lines from the proxy log, all from before the fix:

stream done: finish_reason=length              frames=32736  max_tokens=16384  tools=13
stream done: finish_reason=tool_calls          frames=28888  max_tokens=16384  tools=13
stream done: finish_reason=CLIENT-DISCONNECTED frames=18983  max_tokens=16384  tools=13

The first spent the whole 16,384-token budget and got cut off. The third is my phone giving up at 18,983 frames because the turn had gone on too long. Neither of those is a reasoning failure. They're the same latency problem, far enough out on the tail that it turns into an error.

This hurts more on my setup than on most, and the architecture is why. A dense model reads every weight it has for every token it generates, so speed is capped by how fast the GPU can pull those weights out of memory. On the rented 5090 this one runs 91.1 to 97.6 tokens per second single-stream. Call it 95.

A sparse mixture-of-experts model with a few billion active parameters generates several times faster and can afford to think out loud. This one can't. Every reasoning token is about a hundredth of a second, so 15,000 of them is a wait you feel in your hand.

llama-server has a POST /apply-template endpoint. It takes the same body as a chat completion and returns the fully rendered prompt. No generation, no tokens spent, instant.

I'd never used it. Send it your real request, tools and all, and you see exactly what the model sees. It turned a week of guessing into one paste.

Here's what the template had been putting into every request, directly above the function definitions:

Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.

That's good advice for a hard question with one right answer. Sitting on top of a roster of 13 tools, it's an instruction to walk the other twelve before committing to one, every turn, and walking them takes time I can measure. It's like handing somebody a menu and asking them to think hard about whether they'd rather eat somewhere else.

It also sits in a spot no ordinary system prompt occupies, inside the tools block above the function definitions, which is why a polite "be concise" in my own system prompt never made a dent.

Both came out of reading the rendered prompt.

Both <code>reasoning_effort</code> and <code>chat_template_kwargs.reasoning_effort</code> reach the template.

My first instinct was to reproduce it in isolation: 100 calls across 5 conditions, including deliberately vague prompts like "the project seems disorganized, clean it up," and 5,000-token contexts.

Worst case across all 100 was 403 completion tokens, about four seconds. Nothing spiralled, in any condition.

That result was accurate and completely useless, and it took me too long to see why. A first turn has nothing to loop on. The model calls a tool and stops, because no result has come back yet for it to react to. The behavior only exists once tool results are fed back in. You can't test an agent with single-shot calls. Starting with a real loop would have saved a day.

So I rebuilt it as a real agent loop. Same tasks, same synthetic tool results every run, 9 loops per arm, converted at 95 tok/s:

                        xhigh (default)       low + custom
  median per turn     2,593 tok / 27.3s   1,128 tok / 11.9s
  worst single turn   2,889 tok / 30.4s     833 tok /  8.8s
  max total           4,782 tok             1,469 tok
  concluded on its own       1 of 9             4 of 9

Fifteen seconds back on a median turn, and a session is many turns. The 1-of-9 figure isn't a correctness result. A model that keeps going after it has what it needs produces the same work later, and in an agent loop that stacks: eight turns where three would do is two more minutes of spinner for an answer that was ready at turn three.

One caveat, because I don't want anyone quoting the wrong figure. The synthetic tool results were trivial and identical on every run, which inflates the never-concludes rate in both arms equally. The comparison holds. 1 in 9 is not a real-world termination rate.

A benchmark on synthetic tool results tells you the direction, not the size. So I ran the proxy with the injection on through an evening of ordinary Nymbus work.

First-turn thinking had been averaging around 10,000 tokens with peaks near 15,000. After the change, the peak was 8,000. That's 1m 13s handed back on the worst first turn, and the tail is where all the damage lived: it's what hit the 16,384 cap and what dropped the phone at 19,000 frames.

Quality was unchanged. I went looking for a drop and couldn't find one. Same answers, same tool choices, arriving a minute or more sooner. Nothing got traded for this.

xhigh doesn't make the model smarter. It makes it deliberate longer, and on hardware where every token costs real time, that deliberation is the bill. I was paying two and a half minutes a turn for thoroughness I never asked for and couldn't tell apart from the cheaper version.

When a local model feels slow with tools, render the prompt before you change anything else. POST /apply-template costs nothing and shows you what you're really asking for. Half the time the answer is a sentence somebody else wrote.