Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 beats Claude in our benchmarks

semgrep.dev

Post reply on HN
Print view View original

111–120 of 559 posts

Re: GLM 5.2 beats Claude in our benchmarks

#111

Post by lenerdenator »

rgbrenner wrote: post #19
solenoid0937 wrote: post #3

GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months. Not that it would make any sense.

If that happens it'll be an absolute disaster. Imagine a scenario where Anthropic and OpenAI prohibit most US companies from using their latest models because of safety.. And meanwhile attackers use equivalent open source models to attack US companies. Any prohibition on open source models will do nothing to fix the problem.. since attackers will never feel bound to the law. All advanced models must be available for…

It'd be less about "safety" and more "we've spent trillions developing these AI tools only to have the Chinese, once again, copy them and offer them for pennies on the dollar, and no one seems to care about the impact that has on the long-term sustainability of this sector of the American economy as a whole, so we're yanking the models."

Re: GLM 5.2 beats Claude in our benchmarks

#112

Post by KetoManx64 »

JumpCrisscross wrote: post #106
KetoManx64 wrote: post #99

Earlier quoted context omitted.

As an individual I do not need the whole model. I don't need the model to have knowledge of the rain history of Algeria nor how many colors are in the Russian flag. Once they start trimming down the excess and making them field focused they will run just fine on people's individual devices.

> I do not need the whole model. I don't need the model to have knowledge of the rain history of Algeria nor how many colors are in the Russian flag Isn’t the performance gap between quantized and full models indicative that even if you aren’t using it directly, the model knowing the colors in the Russian flag does have something to do with the intelligence you demand?

Do quantized models specifically prune out specific knowledge? I think they just compress things down but they're still in there. You'd most likely need to do that when you're doing the initial model training, but I'm not expert.

Re: GLM 5.2 beats Claude in our benchmarks

#113

Post by HAL3000 »

sertsa wrote: post #101
noncoml wrote: post #96

Earlier quoted context omitted.

[flagged]

Its an editor: https://zed.dev/

Just FYI, this question was a quote from Pulp Fiction, the other commenter (mdre) replied also with a quote, that was an answer to this question in the movie.

Re: GLM 5.2 beats Claude in our benchmarks

#114

Post by vlian2088 »

oa335 wrote: post #82
vlian2088 wrote: post #9

Earlier quoted context omitted.

>Thirdly it compares to GPT 5.5 mythos is 1000% to run inference on a <10% better model would've been very damning.

> it costs >1000% to run inference do you have a source for this claim? i thought LLM providers earn high margins from inference (charged by token). is this no longer the case?

if a $6000000 cabinet can generate 10000/s tokens of Opus but only 1000/s tokens of Mythos, then Mythos costs 1000% to run no matter the markup.

no one has a source, because no one knows closed model parameter counts. we have only heuristics which strongly indicate that Mythos is simply a big fucking model that any other lab could make an equivalent of.

Re: GLM 5.2 beats Claude in our benchmarks

#115

Post by mrandish »

skissane wrote: post #85
solenoid0937 wrote: post #3

GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months. Not that it would make any sense.

> GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months. I’m sceptical they could find the legal framework to do this even if they wanted to They have legal authority to (a) prevent export of US goods/services; (b) ban imports of physical goods; (c) ban transactions (including purchasing services or license agreements) with foreign…

> I’m sceptical they could find the legal framework to do this even if they wanted to

I agree, my only caveat is that the current administration has shown it's willing to go beyond aggressive regulatory interpretations to questionable and outright implausible interpretations. As we've seen recently, the federal courts and SCOTUS are overturning most of these but that can take a year or more to resolve. The one positive light is they seem to push the hardest on certain culture war issues (immigration, voting, districting, etc). AI doesn't seem like a core hot button issue for the White House and there is a strong pro-AI / business faction.

Re: GLM 5.2 beats Claude in our benchmarks

#117

Post by lenerdenator »

aussiegreenie wrote: post #45
solenoid0937 wrote: post #3

GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months. Not that it would make any sense.

The Americans may ban the use of the Chinese models in America. But like the Chinese car ban, everyone else will use them.

That's not necessarily a good thing for everyone else, mind.

Yes, you get your free model, but the cost of this is not developing your own capability and tying your fate to a country which may or may not have your best interests as a nation in mind.

This is just the deindustrialization that occurred in my home region (the American Midwest) playing out on a global scale in different sectors. It was originally driven by the Japanese, who, to their credit, acted more as partners than competition. Eventually that desire for larger margins went to China, and now you basically can't build anything of consequence without at least some Chinese parts, because there's "no economic case" for it. This means that you have to play Beijing's game if you want access to any sort of modern market.

You see this happening with Volkswagen's restructuring, next you'll see it with non-American, non-Chinese AI.

Re: GLM 5.2 beats Claude in our benchmarks

#118

Post by happycube »

mkagenius wrote: post #84
raincole wrote: post #44

Earlier quoted context omitted.

Yes, and the article author is fully aware of that. Thank you for pointing out this small mistake though.

It looks like the author is specifically avoiding model's name, because results are really weird. Opus 4.8/4.7 scored 28% Opus 4.6 score 37% So the author thought as let's not get into that just write Claude.

Not weird at all, given the variance in Opus' quality over the last few months.

wild guess - I wouldn't be surprised if Opus 4.6 was run quantized for a while, and 4.7/4.8 have QAT for that nerfed size.

Re: GLM 5.2 beats Claude in our benchmarks

#119

Post by cmrdporcupine »

TurdF3rguson wrote: post #105
cmrdporcupine wrote: post #100

I like GLM 5.2... ish. It's ok. I'd be mostly fine switching to it. I just can't find a cost effective way to do that. z.AI's coding plan is both overpriced and unreliable. ollama's is also overpriced. Paying by the token for it on openrouter etc is more expensive than just having a Codex or Claude coding plan. If you have to pay by the token, it's clearly cheaper. It's not competitive with a coding plan though.

It also means giving up vision which I don't know how I would deal with. I think I would prefer a weaker model with vision than a stronger without.

If you using opencode or similar you can just temporarily switch models -- in the same session -- to something that has vision and have it look at your image. And then switch back.
Post reply on HN