Hopefully i get a macbook pro soon enough to run some small or medium sized LLMs
[deleted]
GLM 5.2 beats Claude in our benchmarks
31–40 of 559 posts
Re: GLM 5.2 beats Claude in our benchmarks
#32It reads like an ad. Secondly these are "just" IDORs, arguably the easiest class of vulnerabilities. Thirdly it compares to GPT 5.5 and Opus 4.8. No, we don't have Mythos at home.
Technically we don't have Mythos at all? You guys have access. This tells me we have Opus at home (open weights).
Re: GLM 5.2 beats Claude in our benchmarks
#33It reads like an ad. Secondly these are "just" IDORs, arguably the easiest class of vulnerabilities. Thirdly it compares to GPT 5.5 and Opus 4.8. No, we don't have Mythos at home.
> Thirdly it compares to GPT 5.5 and Opus 4.8.
> No, we don't have Mythos at home.
That's still useful. To paraphrase the kids these days, GLM5.2 is in the room with us, today. Mythos is not. And for us in the EU, it's even more complicated, as Mythos might be with us in the room one day, and go poof the next day, on the whims of political entities that we have 0 control over.
Knowing where open, accessible, local models are is important. We know they're behind. But there comes a time when "good enough" is useful. Even if they're "just IDORs" today, and even if they're behind SotA today.
As someone else said above, GLM5.2 (and other models in the same tier like kimi, dsv4, etc) is / are slowly becoming "good enough" to assist in automated repo prepare work (download, install, test, edit, re-test, etc). And that translates in RL traces ready to be trained into the next generations. That might be more important than x% behind on benchmarks.
Re: GLM 5.2 beats Claude in our benchmarks
#34Hopefully i get a macbook pro soon enough to run some small or medium sized LLMs
Same, but so much $$
Re: GLM 5.2 beats Claude in our benchmarks
#35Earlier quoted context omitted.
Or we use the models to work on fixing vulns and stop over-blowing the doom scenarios. Gotta save the kids and kill the terrorists though! I'm for making software better instead of banning it based on what the rich and powerful claim. I suspect the real fear is that open weight models undermine the financials and token prices they thought were going to pay off their ludicrous spending because they have all raced and…
> making software better instead of banning it That would be the rational thing to do. > financials and token prices I do not think the government thinks this deeply. Market manipulation might be a rational, if unethical reason to ban open source models. But this admin banned Anthropic models to "own the libs." They will continue to ban what they want for whatever reason they want. I don't think those reasons will be…
Yeah, the current admin is reactionary, they appear to put little thought in, or at least disregard input they dislike. I don't think Ant's ban was about "owning the libs" as much as it was asserting dominance over someone who spoke up counter to the admin's aims and claims. They do listen to money, which is where I see Big Ai paying for executive orders (because the admin forgot what it means to compromise as part of legislating for all americans).
Re: GLM 5.2 beats Claude in our benchmarks
#36> [...] beating Claude Code (32%) at roughly $0.17 per vulnerability found Claude Code is an agent harness, not an LLM. Claude is a brand (or group of LLMs), not an LLM.
Claude code it's the only way to get access to the actual amortized cost of running a Claude-scale model. The consumer non-enterprise API is extremely expensive (with increasing marginal costs for the user and fat profit margins for Anthropic). If you want to approximate a State level attacker's cost where they can have the model on their own hardware, Claude Code is probably the best guess at the amortized cost.
Re: GLM 5.2 beats Claude in our benchmarks
#37Apparently GLM 5.2 is 753B parameters [1], what kind of hardware are people using to run this locally?
Re: GLM 5.2 beats Claude in our benchmarks
#38You can launch GLM-5.2 in Opencode using Nemesis8: https://github.com/DeepBlueDynamics/nemesis8#nemesis-8 After installing, do a `n8 build` to build the image, then `n8 --danger --provider opencode interactive` to launch it in a container. Signup for GLM-5.2 here: https://z.ai
One can also try https://neuralwatt.com using it in opencode.
I think they give $5 trail credits to test with any of the open weight models.
Re: GLM 5.2 beats Claude in our benchmarks
#39> Constant: the IDOR dataset (the same real, open-source applications we've used in prior research) ...
What we're they? Also, wouldn't one expect a more recently released coding agent (with a more recent knowledge cut off) to perform better because they have access to more knowledge about vulns in these OSS projects, and even possibly have knowledge of your own "prior research"?
Re: GLM 5.2 beats Claude in our benchmarks
#40Apparently GLM 5.2 is 753B parameters [1], what kind of hardware are people using to run this locally? [1] https://huggingface.co/zai-org/GLM-5.2
follow antirez - https://x.com/antirez/status/2071173841175363905?s=20