Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM-5.2 is a step change for open agents

interconnects.ai

Post reply on HN
Print view View original

61–70 of 240 posts

Re: GLM-5.2 is a step change for open agents

#61

Post by mlmonkey »

Here are the numbers from their bar chart:

    1. SWE-bench Pro
    Model Score (%)
    GLM-5.2 62.1
    GLM-5.1 58.4
    Claude Opus 4.8 69.2
    GPT-5.5 58.6
    Gemini 3.1 Pro 54.2

    2. Terminal-Bench 2.1
    Model Score (%)
    GLM-5.2 81.0
    GLM-5.1 63.5
    Claude Opus 4.8 85.0
    GPT-5.5 84.0
    Gemini 3.1 Pro 74.0
    
    3. NL2Repo
    Model Score (%)
    GLM-5.2 48.9
    GLM-5.1 42.7
    Claude Opus 4.8 69.7
    GPT-5.5 50.7
    Gemini 3.1 Pro 33.4
    
    4. DeepSWE
    Model Score (%)
    GLM-5.2 46.2
    GLM-5.1 18.0
    Claude Opus 4.8 58.0
    GPT-5.5 70.0
    Gemini 3.1 Pro 10.0
    
    5. ProgramBench
    Model Score (%)
    GLM-5.2 63.7
    GLM-5.1 50.9
    Claude Opus 4.8 71.9
    GPT-5.5 70.8
    Gemini 3.1 Pro 39.5
    
    6. MCP-Atlas
    Model Score (%)
    GLM-5.2 77.0
    GLM-5.1 71.8
    Claude Opus 4.8 77.8
    GPT-5.5 75.3
    Gemini 3.1 Pro 69.2
    
    7. Tool-Decathlon
    Model Score (%)
    GLM-5.2 48.2
    GLM-5.1 40.7
    Claude Opus 4.8 59.9
    GPT-5.5 55.6
    Gemini 3.1 Pro 48.8
    
    8. Humanity's Last Exam
    Model Base Score (%) Score w/ Tools (%)
    GLM-5.2 40.5 54.7
    GLM-5.1 31.0 52.3
    Claude Opus 4.8 49.8 57.9
    GPT-5.5 41.4 52.2
    Gemini 3.1 Pro 45.0 51.4
Seems to be handily beating Gemini 3.1 Pro. What _is_ Google DeepMind doing (other than bleeding talent to A\ ) ?

Re: GLM-5.2 is a step change for open agents

#62

Post by fbrncci »

jerojero wrote: post #3

Open weight models from Chinese labs tend to be significantly cheaper. I think theyre absolutely needed. I can't afford 200 USD a month for personal use of coding AI, and I don't think such prices are reasonable for most of the world economy anyway. Not to mention US firms might be giving their employees a lot more than that. It's increasingly feeling, to me, that theres a gap building up between haves and have nots.…

You made me realize something. I routinely spend upwards of 500$ per month on LLMs for coding (expensed towards clients). However I live in a place where 500$ is around the avg. salary. I’m lucky that I know my way around western clients. Clients who pay these expenses and are happy to work with me because I am still about 50% cheaper than local talent in EU/US, while my salary at home converts to an upper class income at the highest tax bracket.

Which of course causes some unfairness on both ends. Nobody here can compete with me. I often use left over tokens on local client projects; which despite lower pay, still pays off because they now take hours not days or weeks to complete. And nobody in the local clients talent pool can compete with me; unless they charge about half the market rate.

Take away my 500$ monthly grant; and I’d be more or less screwed. Better open models will more or less start to reduce this advantage. It’s not like I positioned myself here on purpose. But it’s definitely a „right place, right time“ situation.

Re: GLM-5.2 is a step change for open agents

#64

Post by buzzin__ »

briga wrote: post #44
Balinares wrote: post #2

I can't help wondering what kind of models we'll see coming out of China once it gets its own chip fabs up and running. Right now it sounds like the US's export ban is not slowing them down a whole lot.

With subsidization from the Chinese government they will probably be equal to or better than the models here. I mean, have you looked at the author list of any given AI paper published within, say, the past 5 years? I wouldn't be surprised if half or more AI researches are from China.

Can you compare the amount to the USA subsidization? Which one is bigger? Per Capita? Per unit of economic growth achieved?

Re: GLM-5.2 is a step change for open agents

#65

Post by arcanemachiner »

timcobb wrote: post #35
gandreani wrote: post #31

Earlier quoted context omitted.

I use both the openai subscription and the opencode go subscription. I use the go subscription for my personal work and the openai subscription for my consulting work. The differences between the models are minimal, but I usually stick with gpt-5.4-mini, gpt-5.4, mimo-pro-2.5, deepseek-v4-pro. These latter ones have way more usage than even using 5.4-mini so I tend to use them in personal projects for that reason. My…

Thanks. I was looking at open code go yesterday and I couldn't figure out if the base pricing is including usage or if that's just base pricing and then you have to pay for usage too. How does it work? It is very cheap.

OpenCode Go is a smoking deal IMO. You basically get 6x multiplier on the $10 price since you get $60 worth of usage for $10. And the first month is only $5 so it's even better.

It goes pretty quick, but it's still a great deal. Highly recommended.

Re: GLM-5.2 is a step change for open agents

#66

Post by sosrobahu »

themgt wrote: post #11

I just tested GLM 5.2 out via Z.ai in pi for a little one-off project that was already scoped. It actually did a relatively decent job starting out, and figured important things out from context. But the reasoning traces became increasingly hilarious, with it getting confused and going in loops, doubting itself. I began to feel almost sad, it was like listening to the internal monologue of someone with anxiety disord…

[dead]

Re: GLM-5.2 is a step change for open agents

#67

Post by swader999 »

fbrncci wrote: post #62
jerojero wrote: post #3

Open weight models from Chinese labs tend to be significantly cheaper. I think theyre absolutely needed. I can't afford 200 USD a month for personal use of coding AI, and I don't think such prices are reasonable for most of the world economy anyway. Not to mention US firms might be giving their employees a lot more than that. It's increasingly feeling, to me, that theres a gap building up between haves and have nots.…

You made me realize something. I routinely spend upwards of 500$ per month on LLMs for coding (expensed towards clients). However I live in a place where 500$ is around the avg. salary. I’m lucky that I know my way around western clients. Clients who pay these expenses and are happy to work with me because I am still about 50% cheaper than local talent in EU/US, while my salary at home converts to an upper class inco…

If you are running multiple agents your cost to them should be multiples less what their roi is.

Re: GLM-5.2 is a step change for open agents

#68

Post by IncreasePosts »

matheusmoreira wrote: post #55
Fr0styMatt88 wrote: post #32

Earlier quoted context omitted.

If we can agree that the AI model is at least as capable as a junior engineer or new contractor, how’s that different to saying “software engineering isn’t worth $200 a month”? Has a very race-to-the-bottom feel to it. Though in the grand scheme of it, $200/mo probably isn’t the real price either. Also looking at it not just in a vacuum - paying for a product that can change what you get from under you doesn’t seem g…

Yeah. There's no way to verify what these providers are doing. The real future is running these models at home. Opus level inference on our own hardware would be a dream come true.

How will anyone running home instances be able to compete against people paying some money running much more powerful models on much more powerful hardware?

Re: GLM-5.2 is a step change for open agents

#69

Post by fbrncci »

swader999 wrote: post #67
fbrncci wrote: post #62

Earlier quoted context omitted.

You made me realize something. I routinely spend upwards of 500$ per month on LLMs for coding (expensed towards clients). However I live in a place where 500$ is around the avg. salary. I’m lucky that I know my way around western clients. Clients who pay these expenses and are happy to work with me because I am still about 50% cheaper than local talent in EU/US, while my salary at home converts to an upper class inco…

If you are running multiple agents your cost to them should be multiples less what their roi is.

My costs are 0$ as any token or subscription spend on agents is invoiced as an expense to my clients.

Re: GLM-5.2 is a step change for open agents

#70

Post by selcuka »

devmor wrote: post #9
Dayshine wrote: post #5

Earlier quoted context omitted.

I'm not sure how I'm supposed to get $200 of value out of personal use!

I personally don’t find it that useful for most tasks, but if say, you get paid $50/hr for your work and it saves you more than 4 hours of work in a month, there you go.

Obviously this assumes that you can find 4+ extra hours of $50/hr work every month, or you can work 4 hours less. Neither of these assumptions is correct for people who work for a fixed salary.
Post reply on HN