Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Getting GLM 5.2 running on my slow computer

github.com

Post reply on HN
Print view View original

11–20 of 269 posts

Re: Show HN: Getting GLM 5.2 running on my slow computer

#11

Post by vforno »

walrus01 wrote: post #10

My main question is whether when put into practical use, this can be measured in tokens/second, or more like 1 token per minute... I have seen locally hosted LLM that are as slow as 1 tok/second still be very useful if you give it a project to do something overnight and metaphorically walk away from it, check back with what it has done in 6 or 8 hours. 0.05 to 0.1 tok/s on the other hand, as reported in the URL for t…

In the readme you can see benchmark which everyone with different hardware is running Colibrì, and I have to say I've seen great times! I'm always doing more to improve!

Re: Show HN: Getting GLM 5.2 running on my slow computer

#15

Post by shrinks99 »

Pretty cool! I've also been playing around with GLM 5.2 this week and was equally impressed. At work we're running it locally on some crazy expensive hardware as a test before starting another project so it's great to see people taking this massive FOSS model release and running it on an average machine, even if it's not terribly practical at this point.

Nice work!

Re: Show HN: Getting GLM 5.2 running on my slow computer

#16

Post by vforno »

shrinks99 wrote: post #15

Pretty cool! I've also been playing around with GLM 5.2 this week and was equally impressed. At work we're running it locally on some crazy expensive hardware as a test before starting another project so it's great to see people taking this massive FOSS model release and running it on an average machine, even if it's not terribly practical at this point. Nice work!

Really thanks!!

Re: Show HN: Getting GLM 5.2 running on my slow computer

#17

Post by walrus01 »

vforno wrote: post #11
walrus01 wrote: post #10

My main question is whether when put into practical use, this can be measured in tokens/second, or more like 1 token per minute... I have seen locally hosted LLM that are as slow as 1 tok/second still be very useful if you give it a project to do something overnight and metaphorically walk away from it, check back with what it has done in 6 or 8 hours. 0.05 to 0.1 tok/s on the other hand, as reported in the URL for t…

In the readme you can see benchmark which everyone with different hardware is running Colibrì, and I have to say I've seen great times! I'm always doing more to improve!

I have a 16-core system with 256GB RAM here I could try it with but regretfully it's so old the CPUs aren't AVX2 capable. Otherwise it makes a fairly good llama-server test system for CPU only stuff. Oh well. Time to upgrade (painful to the wallet these days).

Re: Show HN: Getting GLM 5.2 running on my slow computer

#19

Post by vforno »

walrus01 wrote: post #17
vforno wrote: post #11

Earlier quoted context omitted.

In the readme you can see benchmark which everyone with different hardware is running Colibrì, and I have to say I've seen great times! I'm always doing more to improve!

I have a 16-core system with 256GB RAM here I could try it with but regretfully it's so old the CPUs aren't AVX2 capable. Otherwise it makes a fairly good llama-server test system for CPU only stuff. Oh well. Time to upgrade (painful to the wallet these days).

Maybe we can see some integration!
Post reply on HN