Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 beats Claude in our benchmarks

semgrep.dev

Post reply on HN
Print view View original

291–300 of 559 posts

Re: GLM 5.2 beats Claude in our benchmarks

#291

Post by try-working »

pimeys wrote: post #59

I have taken another look on these open models after the fiasco of Fable and GPT 5.6 this weekend and... GLM-5.2 truly is a good workhorse model for daily programming. I consider myself a heavy user of LLMs and a seasoned developer. A typical session for me with GPT is usually over a hundred dollars... This weekend I programmed a matrix bot with encryption and a Rust agent with some tools. Because I need one and Open…

Have you tried using DeepSeek V4 Pro instead? It will be cheaper and faster than GLM.

Re: GLM 5.2 beats Claude in our benchmarks

#292

Post by TurdF3rguson »

gazpachotron wrote: post #153
cmrdporcupine wrote: post #119

Earlier quoted context omitted.

If you using opencode or similar you can just temporarily switch models -- in the same session -- to something that has vision and have it look at your image. And then switch back.

Or create an agent or subagent that just looks at images, and specify a vision model for that agent.

I don't see how that helps, I would still need to somehow get the image into the coding model's context.

Re: GLM 5.2 beats Claude in our benchmarks

#293

Post by andai »

Most interesting things to me from their benchmarks:

GPT does way worse than Opus without their harness, but better with it.

Opus 4.7 and 4.8 do way worse than 4.6. (Intentional nerfing?)

Would have been interesting to see GLM in the custom harness.

Would also be interesting to run GLM in Claude Code, which it has presumably been fine tuned on.

Re: GLM 5.2 beats Claude in our benchmarks

#294

Post by jfaat »

ssk42 wrote: post #272
jfaat wrote: post #262

Earlier quoted context omitted.

> but if you only want to use the best model available, it isn't there yet I'm trying to wrap my head around exactly why so may people seem to want the best model available when it has recently become clear that most halfway decent models can write damn good code for a fraction of the price. And the frontier models get nerfed constantly so you with open weight you can get something slightly less performant but way mo…

What is your favorite harness for the open weights?

We built our own and aren't done open sourcing it but before that I got to a really good place with opencode plus some custom agents, pi family is good too although I haven't used it as much. We made an agent to design a spec, one to implement by dispatching subagents, one to validate against the plan, things like that. All of this helps claude/gpt too IME. For open models it has helped them stay out of loops (e.g. Kimi's but WAIT) and for frontier it helps them stay on task and not invent bloated patterns

Re: GLM 5.2 beats Claude in our benchmarks

#295

Post by vagab0nd »

AussieWog93 wrote: post #219
dakolli wrote: post #55

Earlier quoted context omitted.

8 X RTX6000. It will run you around 80-100k to get started with a model at this size with decent tps.. Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. For $100k you could run this model 24/7 through open router with 10 concurrent sessions at 50tps for a decade and have money left over for a vacation. There's no point in investing this type of mon…

>Don't worry though, open source evangelists will tell you that these will be running on your phone in the next 3 years. Not sure if you're being sarcastic, but I can run a quantised version of Gemma or Qwen on my 16GB M1 Macbook Pro that beats GPT-4 from 2023 hands-down. I wouldn't be surprised if, in another 3 years, you'd be able to run something as powerful as Opus 4.5 or GLM-5.2 on standard consumer hardware - s…

For most tasks, I don't value the LLMs based on their absolute capabilities. I wouldn't want to use GPT-4 today even if it's free.

Re: GLM 5.2 beats Claude in our benchmarks

#297

Post by hedora »

SV_BubbleTime wrote: post #146
CamperBob2 wrote: post #83

Earlier quoted context omitted.

It is so easy to spend $100K on a pickup truck these days, it's not even funny.

Factory F350 Platinum is at least 90k sticker.

Yet Ford claims it is impossible to sell any pickups for > $60K, so they killed the lightning.

I assume (since they claim they are selling the batteries to AI data centers), they’ll produce some sort of EV >= F150 once the bubble pops, and we get a new president.

Re: GLM 5.2 beats Claude in our benchmarks

#298

Post by onoesworkacct »

SwellJoe wrote: post #93

I added GLM 5.2 to my security bug hunting benchmark when it came out, and found it to be a good performer, but not the best open model. The benchmark tests whether models can find bugs Mythos found. The best open models in the initial benchmark were DeepSeek V4 Pro or MiMo 2.5 Pro. But it turned out MiMo got lucky, it's performed worse on almost every test I've done since, while DeepSeek has consistently been among…

could mimo have scraped the mythos findings already? it's very recent

Re: GLM 5.2 beats Claude in our benchmarks

#299

Post by kordlessagain »

generichuman wrote: post #145
kordlessagain wrote: post #2

You can launch GLM-5.2 in Opencode using Nemesis8: https://github.com/DeepBlueDynamics/nemesis8#nemesis-8 After installing, do a `n8 build` to build the image, then `n8 --danger --provider opencode interactive` to launch it in a container. Signup for GLM-5.2 here: https://z.ai

You can use GLM in OpenCode with a z.ai subscription by default as well. Also it'd be good if you mentioned you were involved with nemesis8.

I think it would be good not to suggest someone run a new Chinese agent on their bare metal.

When I posted the comment I was both the first commentor as well as the first person to upvote the submission. That matters. My name is ALSO on the open source repo that allows Opencode to be run in a container.

That's transparency, maybe not here, but on a clickthrough to Github it is immediately obvioius.

Re: GLM 5.2 beats Claude in our benchmarks

#300

Post by wahnfrieden »

pimeys wrote: post #59

I have taken another look on these open models after the fiasco of Fable and GPT 5.6 this weekend and... GLM-5.2 truly is a good workhorse model for daily programming. I consider myself a heavy user of LLMs and a seasoned developer. A typical session for me with GPT is usually over a hundred dollars... This weekend I programmed a matrix bot with encryption and a Rust agent with some tools. Because I need one and Open…

Why are you spending on API for GPT coding instead of stacking 20x subs and using codex-lb?
Post reply on HN