Live data from Hacker News

GLM 5.2 beats Claude in our benchmarks

semgrep.dev

91–100 of 559 posts

Re: GLM 5.2 beats Claude in our benchmarks

#91
post #58
post #55

Earlier quoted context omitted.

8 X RTX6000. It will run you around 80-100k to get started with a model at this size with decent tps.. Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of mon…

you can however, have fun with it. oil workers buy 100k trucks they do not-much with. why not a 100k in computer?

Because car loans can’t be used to buy computers

Re: GLM 5.2 beats Claude in our benchmarks

#93
I added GLM 5.2 to my security bug hunting benchmark when it came out, and found it to be a good performer, but not the best open model. The benchmark tests whether models can find bugs Mythos found. The best open models in the initial benchmark were DeepSeek V4 Pro or MiMo 2.5 Pro. But it turned out MiMo got lucky, it's performed worse on almost every test I've done since, while DeepSeek has consistently been among the best performers and its extreme caching performance makes it cheaper than just about anything, including much smaller models.

https://swelljoe.com/post/will-it-mythos/

Also of note, I found giving models access to the open source semgrep as a tool makes some perform worse and none perform better, though it's plausible there's a way to wire it up in a harness that presents useful information to the model without the model having to know how to use it (my theory is that semgrep isn't heavily represented in the training data, so you're asking the model to do two things at once: figure out how to use semgrep and find security bugs, and both tasks suffer for the lack of focus...most small models, and some big models, can't do that well).

Edit: But, also, more testing is ongoing. I suspect GLM 5.2 will also be a consistently strong performer. It seems to excel at most things I've tested on it.

Re: GLM 5.2 beats Claude in our benchmarks

#94
post #55
post #37

Apparently GLM 5.2 is 753B parameters [1], what kind of hardware are people using to run this locally? [1] https://huggingface.co/zai-org/GLM-5.2

8 X RTX6000. It will run you around 80-100k to get started with a model at this size with decent tps.. Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of mon…

Yeah, the neoclouds and hyperscalers are taking massive losses right now, self hosting is basically signing yourself up to do the same. There are philosophical reasons to do so but it’s a terrible economic decision

Re: GLM 5.2 beats Claude in our benchmarks

#95
post #59

I have taken another look on these open models after the fiasco of Fable and GPT 5.6 this weekend and... GLM-5.2 truly is a good workhorse model for daily programming. I consider myself a heavy user of LLMs and a seasoned developer. A typical session for me with GPT is usually over a hundred dollars... This weekend I programmed a matrix bot with encryption and a Rust agent with some tools. Because I need one and Open…

Are you sure fireworks is unquant? It's not listing precision on openrouter like everyone else.

Re: GLM 5.2 beats Claude in our benchmarks

#97
post #55

Earlier quoted context omitted.

8 X RTX6000. It will run you around 80-100k to get started with a model at this size with decent tps.. Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of mon…

Would you be better off pooling that money with some hackerspace group and then setting up shared inference infra, so that way you at least get better utilization?

And before you know it, you invented some openrouter provider from first principles...

Re: GLM 5.2 beats Claude in our benchmarks

#98
post #82

Earlier quoted context omitted.

>Thirdly it compares to GPT 5.5 mythos is 1000% to run inference on a <10% better model would've been very damning.

> it costs >1000% to run inference do you have a source for this claim? i thought LLM providers earn high margins from inference (charged by token). is this no longer the case?

This was just theorised. The leaked OpenAI financials suggest otherwise (because of shady naming of losses)

The only ones who seem to profit are the ones running smaller Chinese models. Even NVIDIA seems to have to "reinvest" their profits into sponsoring companies to buy their cards now.

Re: GLM 5.2 beats Claude in our benchmarks

#99
post #55
post #37

Apparently GLM 5.2 is 753B parameters [1], what kind of hardware are people using to run this locally? [1] https://huggingface.co/zai-org/GLM-5.2

8 X RTX6000. It will run you around 80-100k to get started with a model at this size with decent tps.. Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of mon…

As an individual I do not need the whole model. I don't need the model to have knowledge of the rain history of Algeria nor how many colors are in the Russian flag. Once they start trimming down the excess and making them field focused they will run just fine on people's individual devices.

Re: GLM 5.2 beats Claude in our benchmarks

#100
I like GLM 5.2... ish. It's ok.

I'd be mostly fine switching to it.

I just can't find a cost effective way to do that. z.AI's coding plan is both overpriced and unreliable. ollama's is also overpriced. Paying by the token for it on openrouter etc is more expensive than just having a Codex or Claude coding plan.

If you have to pay by the token, it's clearly cheaper. It's not competitive with a coding plan though.

Post reply on HN