[flagged]
Show HN: Getting GLM 5.2 running on my slow computer
61–70 of 269 posts
Re: Show HN: Getting GLM 5.2 running on my slow computer
#62related and possibly more general purpose https://github.com/t8/hypura
With so many people implementing their own SSD streaming for specific combinations of model+hardware, maybe we should look into upstreaming to antirez/ds4 or llama.cpp...
Re: Show HN: Getting GLM 5.2 running on my slow computer
#63The page has an SSD wear warning [0] I use desktop PCs that I build from components so I can replace the SSD, but what do users with soldered SSD do? Just avoid these applications or forge ahead disregarding the possible early burnout of their storage? They must use external storage as the burner SSD. [0] https://github.com/JustVugg/colibri#ssd-wear-warning
AppleCare.
Even under AppleCare this is a $400 service which for an older macbook costs almost as much as the whole thing. And without Applecare it's not worth fixing at all.
Re: Show HN: Getting GLM 5.2 running on my slow computer
#64I've taken a similar strategy w/ image/video gen at https://github.com/cretz/thinfer (see video branch for a ton of work). Basically I kept needing an inference engine that could stream weights in and out as needed in an LRU manner. So I ended up vibe coding this thing that accepts a `--vram-budget` and stays under it (mostly). It turns out moving mmap'd bytes in and out of VRAM is way cheap compared to compute. Coup…
Wow, I see you managed to fit in so many models (krea, wan, hunyan, etc.). Did you get to build a common harness to run all of them? Which ones stay under your VRAM budget more consistently?
All stay under because I had Claude build the workflow to respect it (text encoding, denoising, vae, etc), there's just a tiny bit of untracked pieces. While there are common interfaces to invoke them (CLI and API/webpage) and they share ops and some pieces, lots of model logic is unique. This is all vibe coded and surely has inaccuracies.
Re: Show HN: Getting GLM 5.2 running on my slow computer
#65The page has an SSD wear warning [0] I use desktop PCs that I build from components so I can replace the SSD, but what do users with soldered SSD do? Just avoid these applications or forge ahead disregarding the possible early burnout of their storage? They must use external storage as the burner SSD. [0] https://github.com/JustVugg/colibri#ssd-wear-warning
From what I understand, the warning is about swap-out during heavy memory use.
You don't need to be superstitious here: disk activity, including writes in particular, can be measured. E.g. `iostat` or `vmstat` on Linux.
Re: Show HN: Getting GLM 5.2 running on my slow computer
#66The page has an SSD wear warning [0] I use desktop PCs that I build from components so I can replace the SSD, but what do users with soldered SSD do? Just avoid these applications or forge ahead disregarding the possible early burnout of their storage? They must use external storage as the burner SSD. [0] https://github.com/JustVugg/colibri#ssd-wear-warning
From what I understand, the warning is about swap-out during heavy memory use. You don't need to be superstitious here: disk activity, including writes in particular, can be measured. E.g. `iostat` or `vmstat` on Linux.
Yes accurate!
Re: Show HN: Getting GLM 5.2 running on my slow computer
#67My main question is whether when put into practical use, this can be measured in tokens/second, or more like 1 token per minute... I have seen locally hosted LLM that are as slow as 1 tok/second still be very useful if you give it a project to do something overnight and metaphorically walk away from it, check back with what it has done in 6 or 8 hours. 0.05 to 0.1 tok/s on the other hand, as reported in the URL for t…
For most projects the more practical solution is to use clouds offering GLM 5.2 for free. 1 token per minute is minuscule compared to their rate limits for free usage.
Re: Show HN: Getting GLM 5.2 running on my slow computer
#68Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128GB instead of my 64GB...), native Metal kernels, and RAM-only native compressed KV. Happy to put on GitHub when it's ready.
Followed you on GitHub to get notified when you are!
Re: Show HN: Getting GLM 5.2 running on my slow computer
#69Working on something similar targeting macOS on Apple Silicon, Unsloth split GGUF, compressed partial residency in unified memory (would make more sense on 128GB instead of my 64GB...), native Metal kernels, and RAM-only native compressed KV. Happy to put on GitHub when it's ready.
I will be delighted to try. I have a 128gb macbookpro m4 waiting for this.
Re: Show HN: Getting GLM 5.2 running on my slow computer
#70This is something that would benefit from Intel Optane memory. Too bad it was killed at the time.