How to reconcile that with the recent, highly upvoted, article titled: "The gap between open weights LLMs and closed source LLMs"?
What explains it?
Is TFA lying? Is the most upvoted comment here lying?
GLM 5.2 beats Claude in our benchmarks
121–130 of 559 posts
Re: GLM 5.2 beats Claude in our benchmarks
#122Apparently GLM 5.2 is 753B parameters [1], what kind of hardware are people using to run this locally? [1] https://huggingface.co/zai-org/GLM-5.2
follow antirez - https://x.com/antirez/status/2071173841175363905?s=20
Re: GLM 5.2 beats Claude in our benchmarks
#123Anthropic is saying other models were good at detecting vulnerabilities, where Mythos excelled was in creating functional exploits for them. This article only talks about detecting vulnerabilities, so it's unclear if it's a true Mythos equivalent.
It seems "Mythos is really good at finding vulnerabilities" has been what people took away from the Project Glassing announcement, which makes sense. Unfortunately for Anthropic, most seem to have forgotten the best argument Anthropic had for holding Mythos back from the general public, "it's crazy good at crafting exploits". Then, without that context, the tinfoil hats came out.
Re: GLM 5.2 beats Claude in our benchmarks
#124Earlier quoted context omitted.
They could ban payment processors from processing payments to any hosts of GML 5.2, despite the open weights the vast majority of people will be using cloud providers to get access since it is to heavy to host for 99% of people. This would be extremely heavy handed and probably end up accelerating the loss of the virtual US monopoly of payment network. The reast of the world isn't going to let the US dictate that onl…
> They could ban payment processors from processing payments to any hosts of GML 5.2 Can they actually though? Do they have legal authority to tell a payment processor that it has to block transactions of a legal US company, just because the company is hosting a Chinese-developed open source model? I’m sceptical And what about companies (e.g. AWS) that let you “bring your own model”?
It would be extremely heavy handed but the administration has sanctioned the International Criminal Court judges such that they basically have no access to the Wests modern financial system. I think domestic US providers would have to deal with different ways but someone like Herzner could easily be cut off from the financial system if the administration doesn't feel that they are adequately blocking the model
Re: GLM 5.2 beats Claude in our benchmarks
#125GLM export controls incoming? I predict Commerce will force OpenRouter, HuggingFace to take some open models down within the next few months. Not that it would make any sense.
>GLM export controls incoming? US imposing export restrictions on a model from China?
They can easily issue an order for any American company to stop hosting/serving the models. If the model was a threat to national security because of its capabilities then a lot of other countries would follow, including China. No nation will allow some vibe coder with a rogue AI to pose a threat to their systems.
The reason GLM-5.2 hasn't been banned is that despite these cherry picked use cases, GLM-5.2 isn't even close to Opus in all use cases. These vibe benchmarks are ran by companies that are not part of the cyber services offered by Anthropic and OpenAI where they can use the models without the safeguards and refusals so their actual cyber capabilities can be utilized.
These guys that wrote the article compared a gimped Opus to GLM-5.2, knew full well it's misleading, and got the clicks regardless. They don't have enough clout to be a part of something like Project Glasswing, GPT Cyber, etc.
Re: GLM 5.2 beats Claude in our benchmarks
#126[deleted]
Re: GLM 5.2 beats Claude in our benchmarks
#127Earlier quoted context omitted.
As an individual I do not need the whole model. I don't need the model to have knowledge of the rain history of Algeria nor how many colors are in the Russian flag. Once they start trimming down the excess and making them field focused they will run just fine on people's individual devices.
> I do not need the whole model. I don't need the model to have knowledge of the rain history of Algeria nor how many colors are in the Russian flag Isn’t the performance gap between quantized and full models indicative that even if you aren’t using it directly, the model knowing the colors in the Russian flag does have something to do with the intelligence you demand?
Quantizing is one thing. But in general it's self-evident that training the model on information that is irrelevant to your use case does not necessarily improve ability, otherwise you'd have AGI just from reinforcing your model on memorizing the first 10^50 digits of pi.
Likewise, LLMs do not violate the laws of information theory, and therefore the only way to encode X amount of information in Y amount of bits where X > Y is by performing what is effectively lossy compression, and as X grows larger relative to Y the compression ratio must change to lose ever more information.
Yes, for the sake of making chatbots that are "conversational" in that they can interpret natural language as input and produce code as output you can easily benefit in incidental and unintuitive ways by training it on more natural language text. But for a given fixed parameter size, it's possible to produce a better model for a specific task by selectively not muddying its training set in the first place with things that are likely irrelevant to the task.
Re: GLM 5.2 beats Claude in our benchmarks
#128Earlier quoted context omitted.
ASML complies as an ally, why would China comply? The weights are already available and downloaded, is it going to be a crime to have them, run them, make them available? Constitutional rights still exist (I hope)
> it going to be a crime to have them, run them, make them available? Yeah. Illegal numbers.
DeCss was short enough to fit in a t-shirt. Americans are larger these days, but not by enough to fit a decent LLM's weights on an XXXXL shirt, even double sided.
Re: GLM 5.2 beats Claude in our benchmarks
#129I tried GLM many times and it is bad, i have on clue what these people are talking about
Re: GLM 5.2 beats Claude in our benchmarks
#130Earlier quoted context omitted.
Turns out toy drones are more useful in war than multi million dollar planes anyway.
Reaper and Predator are both drones and there’s really no comparison to toy drones in terms of sheer destruction and capabilities in general, the comparison is actually quite apt imo.
Which ones are the ones Ukraine has used to bomb Moscow?