Wut? Was glm 4.7 not just a few weeks ago? I wonder if I will be able to use it with my coding plan. Paid just 9 usd for 3 month.
GLM 4.7 Flash was just a few weeks ago. 4.7 fully I think was a ways further back early December?
GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
21–30 of 540 posts
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#22It's looking like we'll have Chinese OSS to thank for being able to host our own intelligence, free from the whims of proprietary megacorps. I know it doesn't make financial sense to self-host given how cheap OSS inference APIs are now, but it's comforting not being beholden to anyone or requiring a persistent internet connection for on-premise intelligence. Didn't expect to go back to macOS but they're basically the…
>I know it doesn't make financial sense to self-host given how cheap OSS inference APIs are now You can calculate the exact cost of home inference, given you know your hardware and can measure electrical consumption and compare it to your bill. I have no idea what cloud inference in aggregate actually costs, whether it’s profitable or a VC infused loss leader that will spike in price later. That’s why I’m using cloud…
Not concerned with electricity cost - I have solar + battery with excess supply where most goes back to the grid for $0 compensation (AU special).
But I did the napkin math on M3 Ultra ROI when DeepSeek V3 launched: at $0.70/2M tokens and 30 tps, a $10K M3 Ultra would take ~30 years of non-stop inference to break even - without even factoring in electricity. You clearly don't self-host to save money. You do it to own your intelligence, keep your privacy, and not be reliant on a persistent internet connection.
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#23It's looking like we'll have Chinese OSS to thank for being able to host our own intelligence, free from the whims of proprietary megacorps. I know it doesn't make financial sense to self-host given how cheap OSS inference APIs are now, but it's comforting not being beholden to anyone or requiring a persistent internet connection for on-premise intelligence. Didn't expect to go back to macOS but they're basically the…
Yeah that sounds great until it's running as an autonomous moltbot in a distributed network semi-offline with access to your entire digital life, and China sneaks in some hidden training so these agents turn into an army of sleeper agents.
Lol wat? I mean you certainly have enough control self hosting the model to not let it join some moltbot network... or what exactly are you saying would happen?
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#24Earlier quoted context omitted.
> Didn't expect to go back to macOS but their basically the only feasible consumer option for running large models locally. I presume here you are referring to running on the device in your lap. How about a headless linux inference box in the closet / basement? Return of the home network!
Apple devices have high memory bandwidth necessary to run LLMs at reasonable rates. It’s possible to build a Linux box that does the same but you’ll be spending a lot more to get there. With Apple, a $500 Mac Mini has memory bandwidth that you just can’t get anywhere else for the price.
But a $500 Mac Mini has nowhere near the memory capacity to run such a model. You'd need at least 2 512GB machines chained together to run this model. Maybe 1 if you quantized the crap out of it.
And Apple completely overcharges for memory, so.
This is a model you use via a cheap API provider like DeepInfra, or get on their coding plan. It's nice that it will be available as open weights, but not practical for mere mortals to run.
But I can see a large corporation that wants to avoid sending code offsite setting up their own private infra to host it.
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#25Soft launch? I can't find a blog post on their website.
The second sentence from a creative writing prompt: Valerius stood four meters tall—roughly thirteen feet. He was not merely a Space Marine; he was a biological singularity. I'm surprised they still have the emdash and "not x, but y" quirks
distillation is a hell of a drug
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#265.0 flash with native sub-agents released to huggingface.... one can wish right :)
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#27It's looking like we'll have Chinese OSS to thank for being able to host our own intelligence, free from the whims of proprietary megacorps. I know it doesn't make financial sense to self-host given how cheap OSS inference APIs are now, but it's comforting not being beholden to anyone or requiring a persistent internet connection for on-premise intelligence. Didn't expect to go back to macOS but they're basically the…
> It's looking like we'll have Chinese OSS to thank for being able to host our own intelligence, free from the whims of proprietary megacorps.
I don’t know where you draw the line between proprietary megacorp and not, but Z.ai is planning to IPO soon as a multi billion dollar company. If you think they don’t want to be a multi billion dollar megacorp like all of the other LLM companies I think that’s a little short sighted. These models are open weight, but I wouldn’t count them as OSS.
Also Chinese companies aren’t the only companies releasing open weight models. ChatGPT has released open weight models, too.
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#28Earlier quoted context omitted.
> Didn't expect to go back to macOS but their basically the only feasible consumer option for running large models locally. I presume here you are referring to running on the device in your lap. How about a headless linux inference box in the closet / basement? Return of the home network!
Apple devices have high memory bandwidth necessary to run LLMs at reasonable rates. It’s possible to build a Linux box that does the same but you’ll be spending a lot more to get there. With Apple, a $500 Mac Mini has memory bandwidth that you just can’t get anywhere else for the price.
And then only Apple devices have 512GB of unified memory, which matters when you have to combine larger models (even MoE) with the bigger context/KV caching you need for agentic workflows. You can make do with less, but only by slowing things down a whole lot.
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#29Grey market fast-follow via distillation seems like an inevitable feature of the near to medium future.
I've previously doubted that the N-1 or N-2 open weight models will ever be attractive to end users, especially power users. But it now seems that user preferences will be yet another saturated benchmark, that even the N-2 models will fully satisfy.
Heck, even my own preferences may be getting saturated already. Opus 4.5 was a very legible jump from 4.1. But 4.6? Apparently better, but it hasn't changed my workflows or the types of problems / questions I put to it.
It's poetic - the greatest theft in human history followed by the greatest comeuppance.
No end-user on planet earth will suffer a single qualm at the notion that their bargain-basement Chinese AI provider 'stole' from American big tech.
Re: GLM-5: Targeting complex systems engineering and long-horizon agentic tasks
#30Earlier quoted context omitted.
Apple devices have high memory bandwidth necessary to run LLMs at reasonable rates. It’s possible to build a Linux box that does the same but you’ll be spending a lot more to get there. With Apple, a $500 Mac Mini has memory bandwidth that you just can’t get anywhere else for the price.
But a $500 Mac Mini has nowhere near the memory capacity to run such a model. You'd need at least 2 512GB machines chained together to run this model. Maybe 1 if you quantized the crap out of it. And Apple completely overcharges for memory, so. This is a model you use via a cheap API provider like DeepInfra, or get on their coding plan. It's nice that it will be available as open weights, but not practical for mere m…
The needed memory capacity depends on active parameters (not the same as total with a MoE model) and context length for the purpose of KV caching. Even then the KV cache can be pushed to system RAM and even farther out to swap, since writes to it are small (just one KV vector per token).