GLM 5.2 beats Claude in our benchmarks • Hacker News Board

Re: GLM 5.2 beats Claude in our benchmarks

#81

Post by CamperBob2 » Sun, Jun 28, 2026, 10:20 PM UTC

Aurornis wrote: post #74

dakolli wrote: post #55

Earlier quoted context omitted.

8 X RTX6000. It will run you around 80-100k to get started with a model at this size with decent tps.. Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of mon…

> 8 X RTX6000. It will run you around 80-100k to get started 8 x RTX6000 GPUs cost $100,000 alone. You then need to build a system that can support those GPUs with enough PCIe lanes through a PCIe switch. It's going to be $120K to $150K to build or buy a system to run this.

You can run the NV4FP quant with 8x RTX6000 cards at 50-75 tps output, but not (practically speaking) the OEM FP8 version. You will learn more about PCIe than you ever wanted to know.

The real gangstas are running 16x RTX6000s. Too rich for my blood, and the NV4FP quant doesn't seem to be that much worse.

Re: GLM 5.2 beats Claude in our benchmarks

#82

Post by oa335 » Sun, Jun 28, 2026, 10:21 PM UTC

vlian2088 wrote: post #9

danslo wrote: post #6

It reads like an ad. Secondly these are "just" IDORs, arguably the easiest class of vulnerabilities. Thirdly it compares to GPT 5.5 and Opus 4.8. No, we don't have Mythos at home.

>Thirdly it compares to GPT 5.5 mythos is 1000% to run inference on a <10% better model would've been very damning.

> it costs >1000% to run inference

do you have a source for this claim? i thought LLM providers earn high margins from inference (charged by token). is this no longer the case?

Re: GLM 5.2 beats Claude in our benchmarks

#83

Post by CamperBob2 » Sun, Jun 28, 2026, 10:21 PM UTC

Ken_At_EM wrote: post #63

8note wrote: post #58

Earlier quoted context omitted.

you can however, have fun with it. oil workers buy 100k trucks they do not-much with. why not a 100k in computer?

I can't help but ask where this comment came from, you must have some exposure..

It is so easy to spend $100K on a pickup truck these days, it's not even funny.

Re: GLM 5.2 beats Claude in our benchmarks

#84

Post by mkagenius » Sun, Jun 28, 2026, 10:23 PM UTC

raincole wrote: post #44

WithinReason wrote: post #26

> [...] beating Claude Code (32%) at roughly $0.17 per vulnerability found Claude Code is an agent harness, not an LLM. Claude is a brand (or group of LLMs), not an LLM.

Yes, and the article author is fully aware of that. Thank you for pointing out this small mistake though.

It looks like the author is specifically avoiding model's name, because results are really weird.

  Opus 4.8/4.7 scored 28%

  Opus 4.6 score 37%

So the author thought as let's not get into that just write Claude.

Re: GLM 5.2 beats Claude in our benchmarks

#85

Post by skissane » Sun, Jun 28, 2026, 10:23 PM UTC

solenoid0937 wrote: post #3

GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months. Not that it would make any sense.

> GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months.

I’m sceptical they could find the legal framework to do this even if they wanted to

They have legal authority to (a) prevent export of US goods/services; (b) ban imports of physical goods; (c) ban transactions (including purchasing services or license agreements) with foreign firms

But I’m not aware of any legal authority which lets them ban US firms from running a Chinese-developed open source AI model in the United States, if they are at arms length from the vendor, and aren’t using it for government contracts or regulated applications

Possibly they could order HuggingFace/etc to suspend Chinese accounts. But if someone in the US (or a third country) downloads the model from China then reuploads it to a US server, completely independently of the vendor - where is the legal hook to prohibit that?

Re: GLM 5.2 beats Claude in our benchmarks

#86

Post by csjh » Sun, Jun 28, 2026, 10:27 PM UTC

I found it to spiral into complete nonsense a few times when I tested it out, but it's possible that was a bug in the provider

Re: GLM 5.2 beats Claude in our benchmarks

#87

Post by kccqzy » Sun, Jun 28, 2026, 10:30 PM UTC

bArray wrote: post #37

Apparently GLM 5.2 is 753B parameters [1], what kind of hardware are people using to run this locally? [1] https://huggingface.co/zai-org/GLM-5.2

Run quantized versions. https://unsloth.ai/docs/models/glm-5.2

Re: GLM 5.2 beats Claude in our benchmarks

#88

Post by andriy_koval » Sun, Jun 28, 2026, 10:30 PM UTC

mkagenius wrote: post #84

raincole wrote: post #44

Earlier quoted context omitted.

Yes, and the article author is fully aware of that. Thank you for pointing out this small mistake though.

It looks like the author is specifically avoiding model's name, because results are really weird. Opus 4.8/4.7 scored 28% Opus 4.6 score 37% So the author thought as let's not get into that just write Claude.

many people think opus 4.6 was the best

Re: GLM 5.2 beats Claude in our benchmarks

#89

Post by mkagenius » Sun, Jun 28, 2026, 10:31 PM UTC

theteapot wrote: post #39

> Constant: the IDOR dataset (the same real, open-source applications we've used in prior research) ... What we're they? Also, wouldn't one expect a more recently released coding agent (with a more recent knowledge cut off) to perform better because they have access to more knowledge about vulns in these OSS projects, and even possibly have knowledge of your own "prior research"?

One would. But then the results are even weirder as opus 4.6 scored more than opus 4.8 by a huge margin

Re: GLM 5.2 beats Claude in our benchmarks

#90

Post by InvertedRhodium » Sun, Jun 28, 2026, 10:31 PM UTC

dakolli wrote: post #55

bArray wrote: post #37

Apparently GLM 5.2 is 753B parameters [1], what kind of hardware are people using to run this locally? [1] https://huggingface.co/zai-org/GLM-5.2

8 X RTX6000. It will run you around 80-100k to get started with a model at this size with decent tps.. Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of mon…

Depends how much you value privacy and running uncensored models.

Personally, I’m waiting for hardware to hit the secondary market before I buy something to run unquantized models like GLM. But I have no doubt that I will, at some point.