Eleven blacklisted hosts, one bad field

I rent GPUs by the hour to serve a 27B model to my iPhone. For about a week, the pods kept dying. Four minutes in, forty minutes in, never longer than that. I'd blacklist the host, launch somewhere else, and watch the exact same thing happen.

The status always read the same way, and it never made sense:

intended_status:  stopped
status_msg:       "success"

Nothing crashed. The container was healthy right up until it wasn't there anymore. Something stopped it deliberately, reported success, and moved on. It took me four days and eleven blacklisted hosts to work out that the something was me.

Nymbus is my iOS app for talking to local models. Most of the time it points at LM Studio on my Mac. When I want the 27B at a speed that doesn't feel like waiting on a fax, I rent an RTX 5090 on Vast.ai for around $0.38 an hour, run llama.cpp on it, and point the phone at a proxy that forwards to the pod.

The whole thing is a Python launcher. Search the offer list, pick a host, PUT a create request, wait for the port to open, register the endpoint, and hand it to a reaper that kills the pod once I stop using it. Vast has no server-side terminate, so that reaper isn't a backstop. It's the only stop.

The pods that died were always the cheap ones. That correlation was real, and I checked it before acting on it. So I raised the bar. Require Vast-verified hosts. Push the reliability floor from 0.97 up to 0.99. Add a bandwidth minimum so I'd stop landing on hosts that took half an hour to pull a 17GB quant.

It got worse. A verified host in California sitting at 99.6% reliability died on the same schedule. So did a verified one in Denmark. Same signature, same status_msg: "success", same shrug.

The filters also priced me out of my own budget. On a 101-offer sample of single 5090s, seven offers cleared the full stack. The cheapest of those seven was $0.550 an hour. I'd been paying $0.481 for the pods that were already dying.

So now I was spending more money to fail just as fast. That's usually the moment I stop trusting my own reasoning, which is exactly what needed to happen here. The correlation was solid. My reading of it was backwards.

My create request carried a price key. I wrote that code, and I wrote the docstring above it, and the docstring said the field was a ceiling rather than a bid. I'd read that line a dozen times and nodded at it every time.

Vast's create spec says this about price:

Bid price per machine (in $/hour). Only for interruptible instances.

The official CLI agrees from the other direction. Its --bid_price flag is documented as "create an INTERRUPTIBLE instance." Sending that field at all is what makes a rental a bid. There's no separate switch anywhere.

I'd assumed I was covered because my search query asks for type: "on-demand". That parameter only picks which price column the offer listing reports back to me. The create body decides what kind of rental you actually get, and mine had been asking for a bid since the day I wrote it.

One API call against the pod that happened to be running confirmed the whole thing.

Once you know every rental was a bid, the part that made no sense makes complete sense. Vast pauses an interruptible instance if someone places a higher bid, or if an on-demand rental is created for the same resources. Verified machines with fast links and high reliability scores are the ones a paying on-demand renter actually wants. So those are the machines that get claimed back soonest.

Renting a bid instance is like flying standby. You board if there's room, and you're off the plane the moment a full-fare passenger shows up. I was flying standby while paying full fare. Then I "fixed" it by moving myself onto the busiest routes on the board.

The cheap hosts weren't unreliable. They were just less in demand, so nobody came along to claim my seat.

Verification got easy once I knew what to measure. Every interruptible pod that week had died inside 40 minutes. The first on-demand pod ran 5 hours 15 minutes on a full lease, and it was still serving when I shut it down deliberately.

wait_ready() now hard-fails if is_bid comes back True on a fresh instance. One field, one API call, and a class of failure that ate four days becomes an error message on launch.

Deleting price also removed the only spending ceiling that existed at create time. That's a real loss, even though the ceiling had been doing the wrong job. So I added an explicit check: compare the billed dph_base against the quoted rate, destroy the instance if it drifts more than 5%. That isn't paranoia. Instance 48396466 quoted $0.4133 and billed $0.4444, 7.5% over, and the old code accepted it silently.

Finding the real cause made me overcorrect immediately. I cleared the entire blacklist as preemption noise and went looking for other things I'd gotten wrong. Three of them were sitting right there in the same week's logs.

**Read the error string before you theorize about the mechanism.** That one cost the least money and is the easiest to repeat. I had a plain-English explanation of the failure printed on every single poll, and I spent three launches guessing at a number instead.

Different problem, same shape. The model's tool calls had been sloppy in long agent loops for a while. Not wrong, exactly. Endless. It would call a tool, think about the result at length, call another one, and keep going well past the point where it clearly had the answer.

Qwen3.8's chat template defaults reasoning_effort to xhigh. At xhigh it injects a plain string above the tool definitions:

Please think carefully through the task, validate key assumptions,
consider plausible alternatives, and prioritize correctness,
consistency, and clarity

That's decent advice for writing an essay. Sitting on top of a 13-tool roster, "consider plausible alternatives" is close to the worst instruction you can hand a model.

Single-shot testing showed me nothing. I ran 100 calls across 5 conditions and the worst case was 403 thinking tokens, which is completely fine. The failure only shows up once tool results come back in and the model has to decide whether it's finished.

So I measured it in a real loop instead. Nine multi-turn agent loops per arm on the Illinois 5090, identical tasks, identical tool results fed back:

                median total   max total   concluded on its own
  xhigh                2,593       4,782                 1 of 9
  low + custom         1,128       1,469                 4 of 9

Halving the token count is nice. Termination is the real find. At xhigh the model almost never decides it's done. It keeps calling tools until a token cap or a client timeout stops it, and from the outside that reads as bad tool-calling behavior rather than a reasoning setting nobody configured.

The fix: on any request that carries tools, set reasoning_effort: "low" and append a short custom instruction to the system message telling it to call one tool and stop. Chat turns with no tools are left alone and keep full reasoning. Confirmed in production that same evening, first-turn thinking dropped from an average around 10,000 tokens with peaks near 15,000, down to a peak of 8,000. Output quality was unchanged.

One trap worth writing down, because it cost me a test run: medium does nothing at all. The template's if/elif branches handle xhigh and low and nothing else, so medium falls straight through and injects an empty string. low is the only value that actually asks for brevity, and anything outside those three raises a 500.

is_bid was sitting on every instance I ever launched. status_msg prints on every poll. The chat template ships with the model and I can open it in a text editor. None of these needed a new tool or a clever idea. They needed me to look at a field instead of building a theory about what the field probably said.

I spent four days blacklisting other people's hardware over a bug in my own create body. The hosts were fine. My reasoning was the thing that needed a filter.