Fixed three bugs that made Qwen3.5-122B a daily driver on Mac Studio

I spent three weeks debugging why my Qwen 122B setup on an M3 Ultra was taking 3–5 minutes to generate the first token on follow-up messages (despite having a "warm" context).

The root cause wasn't the model, but three specific infrastructure bugs in my serving stack:

1. Prompt Instability: A unique message ID in the system prompt broke byte-exact KV cache matching, forcing a full re-compute every turn.

2. Interrupt Path: Streaming replies weren't persisted when the generation was interrupted, causing history divergence.

3. Checkpoint Poison: A background writer created unmatchable checkpoints that crowded out valid ones, triggering aggressive eviction.

After fixing these, prefill time dropped from minutes to sub-seconds (53k tokens cached, 33 tokens prefilled).

I've open-sourced the fork (qMLX) and a benchmarking tool to verify these numbers. Would love feedback on the hybrid attention caching strategy or any other edge cases I might have missed.

Impressive debugging skills, and thank you for the benchmarks. Now I'm wondering if mlx-engine / mlx-lm have these bugs too.

One minor thing: as you are concerned with honest numbers, the graphs should be logarithmic on the y-axis too (like they are on the x-axis). Otherwise it's hard to see whether the curve is sublinear or linear.

I appreciate the amount of detail in the post, I think it's a useful addition to the space.

That said, I have to read LLM output all day all the time, and I would implore you to take the time to explore your own voice a bit more.

> Two separate things then happened, and it is worth keeping them apart.

Is one of those phrases claude spits out nonstop.

Nonstop. There is too much wholesale reliance on LLMs to generate content. When I use LLMs for scientific writing, I approach it differently: I write the paragraph dirty, then ask an LLM to perform a minor rewrite for "clarity", using Claude's now retired Concise Mode. This has been a great approach for scientific writing. It tends to prevent these overly used turns of phrase, it makes sure that the writing is making the points I want to make, and shortens writing time by cleaning up the dirty edges of my grammar (especially since my writing can tend towards convoluted constructions). More artistic/creative writing, I'd probably not use it at all, because then, it's usually (for me) about rthym and emotional flow.

BTW It IS an effective rhetorical phrase, but given it's ubiquity in Claude's output, I have to avoid it.

Fair enough, nothing more infuriating than the sycophantic way LLMs write.

But I’d definitely be dishonest if I said I’d stop using LLMs to tidy my writing (out of principle or otherwise), my hold on the English language has seriously degenerated over the last eight years since I pivoted into IT from customer facing roles.