How I'd set up LM Studio today

A couple weeks ago a Llama.cpp update landed that moved local inference enough that I noticed without measuring. Then I measured.

Before the update, my go-to model peaked around 57 to 65 tokens per second and decayed as context grew. Long sessions would drift into the mid-50s, which is right around the point where agent loops start to feel like watching a kettle.

After the update, the same model holds 80 to 90 tokens per second across the whole session, with the worst dip deep into context landing near 78. Even the bad moments are faster than every good moment on the old build.

That's the story. If you've bounced off local models because they felt sluggish during real work, it's worth trying again this week.

I'm not chatting with my local model. I'm running loops. Tool call, review, edit, retry, summarize. A single feature iteration might fire 30 messages across two models. At 57 tok/s and degrading, that kind of loop stalls. You feel every pause. You stop trusting the flow to finish, and you end up babysitting it.

At 80-plus sustained, the loop gets out of your way. The model didn't get smarter this month. The wait cost dropped far enough that I stopped opting out of agent workflows, which is a different kind of improvement.

The 80-plus I mentioned is my big-model floor. On the small end, the same update pays off even harder. A 2B dense model on this machine now clears 160 tokens per second. That's fast enough that the reply lands the moment you stop typing.

Small models handle intake, classification, short replies, anything where the user is waiting on a fast answer. The MoE handles the work that actually needs reasoning. Here are two back-to-back measurements from the LM Studio status bar, same machine, same week.

Most of the advice online ignores this. The right model depends on your hardware, and trying to stretch past it is how you end up with 8 tok/s and swap thrash.

Modest hardware, the MacBook Air M-series tier with 16 to 24 GB of unified memory: run one small dense model. I'd pick from Qwen 3.5 4B, Gemma 4 4B, or Nemotron 5 Nano 4B. These are genuinely good for intake, classification, summarization, and short creative work. Coding is okay, not their strength.

Mid-range, 32 to 64 GB of RAM with a dedicated GPU or an M-series Pro or Max: step up to an 8B. Gemma 4 8B, Llama 3.6 8B, or Qwen 3.5 8B all feel noticeably stronger on reasoning and code while staying fast.

Serious workstation, 32 GB-plus unified memory on recent Apple silicon (M2 Max onward) or a high-VRAM GPU: this is where the interesting move is. Skip dense models and run an MoE, specifically Qwen 3.6 35B-A3B. 35 billion total parameters, only about 3 billion active per token. You pay the memory bill of a big model and the speed bill of a small one. Two things to know about the threshold. RAM decides what you can load, so 32 GB is enough to fit a 35B MoE. Chip generation decides how fast it runs, and once you're on M4-class silicon a 32 GB machine will leave a 64 GB older-generation box behind on throughput.

The Developer tab's Server Settings panel is the difference between a model that feels fast and one that doesn't. Here's what I actually toggle.

These aren't magic. They're reasonable defaults for serious work, and LM Studio ships with them off or low for safety. Flip them.

MoE is how you get a big model's breadth at a small model's speed. That's the sentence I kept trying to write for two years before a release earned it.

The intuition: a big model that only lights up the parts it needs for each token. For Qwen 3.6 35B-A3B, the math shakes out like this:

• 35 billion total parameters in memory
• ~3 billion active per token, so throughput feels like a dense 8B
• Reasoning quality closer to the 24 to 30B dense class
• 32 GB on M2 Max or newer is enough to load it

The Discover page pitch flags "stability and real-world utility" and "agentic coding," which is exactly the use case that used to send me back to a cloud API.

Loading and speed are different stories, though. More RAM fits bigger models. A newer chip decides how fast whatever you load actually runs. On the M3 Max I can't clear 45 tokens per second with the same kernel and the same model that hits 80-plus on the M4 Max. Same model, same kernel, nearly double the throughput on newer silicon.

A year ago I'd have told you to keep the serious work on a frontier API and treat local as a nice offline backup. That answer changed two weeks ago.

If you've got modest hardware, a 4B gets you further than you'd expect. If you've got mid-range hardware, an 8B is a real daily driver. If you've got a workstation, Qwen 3.6 35B-A3B is the model to try first.

Then open the Developer tab, flip those settings, and run an agent loop. Not a single-turn chat. The whole thing. That's where the speed bump actually pays you back.