Ranked #1, and three minutes late

Artificial Analysis puts Qwen3.8-27B at 52 on its Intelligence Index, ranked first out of 137 models in its comparison class. That's a 27B set of open weights, level with systems many times its size and with frontier models I can't run at all.

It matches what I see using it. It's consistent with tool calls, which is rare at this size. It holds up on long-running tasks instead of losing the plot three turns in. It runs tests, reads its own output, decides whether that output is right, and writes files. In Nymbus, my iOS app, it drives an agent loop with 13 tools in front of it, and it hasn't needed babysitting.

There's a detail in the Artificial Analysis numbers I keep coming back to. The hosted providers they measure serve this model at 53.8 tokens per second on Alibaba Cloud and 42.6 on DeepInfra. The RTX 5090 I rent by the hour runs it at about 95. Whatever was wrong with my setup, the silicon wasn't it.

It's a dense 27B. Every weight gets read for every token it generates, so speed is capped by memory bandwidth and it's never going to be quick. That's the floor, and I knew about the floor.

What I didn't expect was how far above the floor the wait landed. First-turn thinking traces were running 15,000 tokens, sometimes 18,000, before the model took a single action. Priced in the only unit that matters:

                          5090 @ 95 tok/s   M4 Max @ 28.5 tok/s
  15,000 thinking tokens        2m 37s              8m 46s
  18,000 thinking tokens        3m 09s             10m 32s

Ten and a half minutes of silence on the laptop before the first tool call. Three turns like that back to back is over half an hour for a task that produces a few files. In an agent workflow that's the difference between a handful of minutes and an afternoon.

Here's what turns an annoyance into a real problem. A model doesn't generate at a constant speed. It slows down as the context fills, and every thinking token it produces is permanent weight added to that context for every turn that follows.

Same host, same model, same session, from my own benchmark files:

  short generation  (~1,300 tokens)   116.8 and 115.1 tok/s
  long generation   ( 4,000 tokens)    88.0 and  99.5 tok/s

Roughly a 25% drop from nothing more than producing a longer answer in one turn. My Apple silicon calibration data says the same thing from another angle: Qwen3.6-35B on the M4 Max runs 86 to 94 tok/s early in a generation and tapers to around 70 above 16K of context.

I'm not going to draw a curve through two measurements. The directional version is enough. Speed falls as context grows, I measured about a fifth to a quarter, and 15,000 tokens of thinking per turn is the fastest way to grow context there is. Long thinking makes the model progressively slower at exactly the moment an agent loop needs it most.

There's an irony sitting in the same source I opened with. Running their evaluation on this model took 160 million tokens, against a median of 45 million across the models they test. Their note on it:

very verbose in comparison

Three and a half times the usual token spend to reach a score that ranks first. The run that establishes the intelligence is the same run that documents what reaching it costs.

llama-server has a POST /apply-template endpoint. It takes the same body as a chat completion and returns the fully rendered prompt. No generation, no tokens spent, instant.

I'd never used it. Send it your real request, tools and all, and you see exactly what the model sees. That's where the answer was.

Here's what the template had been putting into every request, directly above the function definitions:

Reasoning effort is set to xhigh. Please think carefully through the task, validate key assumptions, consider plausible alternatives, and prioritize correctness, consistency, and clarity in the final answer.

Good advice for a hard question with one right answer. Sitting on top of a roster of 13 tools, it's a standing instruction to walk the other twelve before committing to one, every turn.

It also occupies a spot no ordinary system prompt reaches, inside the tools block above the function definitions, which is why a polite "be concise" in my own system prompt never made a dent.

Both came out of reading the rendered prompt.

Both <code>reasoning_effort</code> and <code>chat_template_kwargs.reasoning_effort</code> reach the template.

My first instinct was to reproduce it in isolation: 100 calls across 5 conditions, including deliberately vague prompts like "the project seems disorganized, clean it up," and 5,000-token contexts. Worst case across all 100 was 403 completion tokens, about four seconds. Nothing spiralled.

That result was accurate and useless. A first turn has nothing to loop on. The model calls a tool and stops, because no result has come back for it to react to. The behavior only exists once tool results are fed back in.

So I rebuilt it as a real agent loop. Same tasks, same synthetic tool results every run, 9 loops per arm, converted at 95 tok/s:

                        xhigh (default)       low + custom
  median per turn     2,593 tok / 27.3s   1,128 tok / 11.9s
  worst single turn   2,889 tok / 30.4s     833 tok /  8.8s
  concluded on its own       1 of 9             4 of 9

Fifteen seconds back on a median turn, and a session is many turns. One caveat, because I don't want the wrong figure quoted: the synthetic tool results were trivial and identical every run, which inflates the never-concludes rate in both arms equally. The comparison holds. 1 in 9 is not a real-world rate.

A synthetic benchmark tells you the direction, not the size. So I ran the proxy with the injection on through an evening of ordinary Nymbus work.

First-turn thinking had been averaging around 10,000 tokens with peaks near 15,000. After the change, the peak was 8,000. That's 1m 13s handed back on the worst first turn, and the tail is where all the damage lived.

Quality was unchanged. I went looking for a drop and couldn't find one. Same answers, same tool choices, arriving a minute or more sooner.

xhigh doesn't make the model smarter. It makes it deliberate longer, and on hardware where every token costs real time and each turn costs more than the last, that deliberation is the bill. I was paying three minutes a turn for thoroughness I never asked for and couldn't tell apart from the cheaper version.

The fix was never about making the model better. It was already good. The fix was to stop paying a tax that had never bought anything, and unchanged output quality is the proof.