Show HN: Getting GLM 5.2 running on my slow computer • Hacker News Board

Re: Show HN: Getting GLM 5.2 running on my slow computer

#221

Post by throwdbaaway » Fri, Jul 10, 2026, 4:24 PM UTC

tarpitt wrote: post #53

vforno wrote: post #38

Earlier quoted context omitted.

I have a small laptop. If you have more disks available, you could really do some testing. When you have some benchmarks, submit a pull request or issue so we can maybe work on them. We are really happy for contribute!

I have epyc 9654 ES and a 7900 XTX. I was running the numbers, and even if I maxxed out the ram to like 12x32 gig sticks, it would cost me thousands more and I could only run GLM-5.2 at a couple tokens per second at q3. So this project is very promising because it suggests I could get pretty high speed and this CPU/motherboard combination suggests I have a lot of pci bandwidth that is unused. I think another route mi…

If you max out the ram, TG with q3 should be at least 10 t/s. And with dsa, it can still stay close to that number as the context grows.

Re: Show HN: Getting GLM 5.2 running on my slow computer

#222

Post by rhdunn » Fri, Jul 10, 2026, 4:37 PM UTC

lopatin wrote: post #176

After spending way too much time with Fable a few days ago, I noticed a new hallmark of AI generated text is using the word "honest" everywhere, in a somewhat self congratulating way. Some things that give it away to me: - "Honest numbers (WSL2, 12 cores, 25 GB RAM, NVMe via VHDX)" - "an honest peak projection (working set, KV, MTP row, reconstruction buffers) so the kernel OOM-killer never fires." - "Honest caveat f…

It's not goblins, it's honest!

Re: Show HN: Getting GLM 5.2 running on my slow computer

#223

Post by BigTTYGothGF » Fri, Jul 10, 2026, 5:58 PM UTC

Cieric wrote: post #26

I was actually just working on the same thing as this, but I went down the route of mmapping the entire model into memory to avoid the extra ram usage. I also had Claude implement Medusa[1] on the model to try and avoid loading an additional model into memory but still get the benefits of MTP. Currently at a stop light so I can't list everything and I didn't get to read your full post either yet. To expand since I ju…

> Currently at a stop light so I can't list everything and I didn't get to read your full post either yet.

Posting on the Hacker News forum is not worth risking your, or anybody else's, life.

Re: Show HN: Getting GLM 5.2 running on my slow computer

#224

Post by lukas9 » Fri, Jul 10, 2026, 8:06 PM UTC

badatnames wrote: post #208

abe94 wrote: post #207

Earlier quoted context omitted.

I've been seeing a lot of usage of the word `real` from recent fable sessions: - That gap is the real story: 3,873 flows - the real conversion filter types are: - I'll update the breakdown query to include a column for each real type

I have almost completely stopped writing real in my own output as a direct consequence of this. Maybe next year they'll catch up with my alternative wordings and I'll end up switching back

This is what I've noticed lately. I write articles and over the past year or two I had to stop using specific words or change my writing style a bit many times, because people kept asking if it was AI. It just makes me sad when I show someone my work and the first thing they say is "Is this AI"?

Re: Show HN: Getting GLM 5.2 running on my slow computer

#225

Post by walrus01 » Fri, Jul 10, 2026, 11:05 PM UTC

adrian_b wrote: post #197

walrus01 wrote: post #10

My main question is whether when put into practical use, this can be measured in tokens/second, or more like 1 token per minute... I have seen locally hosted LLM that are as slow as 1 tok/second still be very useful if you give it a project to do something overnight and metaphorically walk away from it, check back with what it has done in 6 or 8 hours. 0.05 to 0.1 tok/s on the other hand, as reported in the URL for t…

Now many mini-PCs and desktops are able to read simultaneously from 1 PCIe 5.0 SSD and 1 PCIe 4.0 SSD. This can ensure a reading throughput around 20 GB/s, i.e. 20 times faster than on author's system. With only 1 PCIe 5.0 SSD, the reading throughput is still significantly more than 10 times faster than on author's system. So it is likely that inference speeds around 1 token/s are achievable on something like a NUC m…

I'm curious, where are you seeing M.2 2230 to M.2 2280 size NVME SSE that exceed 4.5 to 5GB/s sequential reads for large files such as a GGUF (likefrom an ordinary ext4fs file system with default options)? The PCI-E 4.0 or 5.0 bus they're attached to might be capable of greater speeds, but the bottleneck is the flash and the flash controller.

The benchmarks I'm seeing for many of them don't really make me think that a pair of consumer grade NVME SSD you could fit in a mini-PC or mini-itx size desktop would, added together, be capable of 20GB/s reads.

Re: Show HN: Getting GLM 5.2 running on my slow computer

#226

Post by pobonin » Sat, Jul 11, 2026, 12:09 AM UTC

I must say, this is incredible, nice work! I've been tinkering with local LLMs for a while now and experimenting with different tech, and I'd never thought it'd be possible to run such a large model on consumer-grade compute. It still feels like these days, you need to buy a $10K DGX Spark or something crazy expensive just to get to that level of performance but with experiments like yours, it looks like there's still a lot of room for improvement and innovation.

What I'd like to see personally is Small Language Models giving similar performance eventually (because do we really need that many hundred billion parameters?). Researchers have shown that a lot of attention heads in the LLMs we're using these days are barely even active, so the MoE architecture seems like a step in the right direction. I still wonder how far we can push things back towards a 1B or 3B parameters model while trying to reach similar performances as double-digit or triple-digit B params models.

Re: Show HN: Getting GLM 5.2 running on my slow computer

#227

Post by Cieric » Sat, Jul 11, 2026, 1:17 AM UTC

BigTTYGothGF wrote: post #223

Cieric wrote: post #26

I was actually just working on the same thing as this, but I went down the route of mmapping the entire model into memory to avoid the extra ram usage. I also had Claude implement Medusa[1] on the model to try and avoid loading an additional model into memory but still get the benefits of MTP. Currently at a stop light so I can't list everything and I didn't get to read your full post either yet. To expand since I ju…

> Currently at a stop light so I can't list everything and I didn't get to read your full post either yet. Posting on the Hacker News forum is not worth risking your, or anybody else's, life.

Being fully stopped at a stop light isn't putting my or anyone else's life in danger. If I have a phone in my hand the car is not moving. I also finished the comment before the light turned green so the phone also was no longer in my hand when I needed to start moving.

Re: Show HN: Getting GLM 5.2 running on my slow computer

#228

Post by fwipsy » Sat, Jul 11, 2026, 2:09 AM UTC

mariopt wrote: post #7

I wonder if you could replicate this in a Colourful GeForce RTX 50-series GPU, they ship it with 2 NVMe drive slots.

I was really curious about it - looks like the extra drive slots are NOT wired to the GPU's VRAM, they're just using PCIE bifurcation to free up some extra lanes for people who plug an x8 GPU into an x16 slot.

Re: Show HN: Getting GLM 5.2 running on my slow computer

#229

Post by brettpro » Sat, Jul 11, 2026, 5:14 AM UTC

abe94 wrote: post #207

lopatin wrote: post #176

After spending way too much time with Fable a few days ago, I noticed a new hallmark of AI generated text is using the word "honest" everywhere, in a somewhat self congratulating way. Some things that give it away to me: - "Honest numbers (WSL2, 12 cores, 25 GB RAM, NVMe via VHDX)" - "an honest peak projection (working set, KV, MTP row, reconstruction buffers) so the kernel OOM-killer never fires." - "Honest caveat f…

I've been seeing a lot of usage of the word `real` from recent fable sessions: - That gap is the real story: 3,873 flows - the real conversion filter types are: - I'll update the breakdown query to include a column for each real type

It's been "mathematical" from Gemini for me. Mathematically verified yaml files for Home Assistant, mathematical confirmation of (incorrect) CLI args for a tool, mathematically indistinguishable nutrient profiles of two ingredients.

Re: Show HN: Getting GLM 5.2 running on my slow computer

#230

Post by fibonachos » Sat, Jul 11, 2026, 5:47 AM UTC

badatnames wrote: post #208

abe94 wrote: post #207

Earlier quoted context omitted.

I've been seeing a lot of usage of the word `real` from recent fable sessions: - That gap is the real story: 3,873 flows - the real conversion filter types are: - I'll update the breakdown query to include a column for each real type

I have almost completely stopped writing real in my own output as a direct consequence of this. Maybe next year they'll catch up with my alternative wordings and I'll end up switching back

I’ve started learning into the various AI-isms in my prompts. Most recently, em dashes to chain related information in a single sentence.

“Something something main point — but also pay attention to this related thing — finish main point.”

Anecdotally, it seems to produce more consistent results when I “speak its language”.

On the output side, I’ve become much more aggressive about trimming and editing any prose it produces that I intend on sharing. Both for my own benefit to improve comprehension, but also because dumping the raw, verbose output on my colleagues feels lazy and disrespectful of their time.