Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 and the coming AI margin collapse

martinalderson.com

Post reply on HN
Print view View original

31–40 of 495 posts

Re: GLM 5.2 and the coming AI margin collapse

#31

Post by cogman10 »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them.

The UX is the same regardless the provider. You send in a prompt, it spits back an answer.

In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with. The transition is basically updating your current harnesses to know about the other models.

I think the comparison most apt is the rise of AMD. Sure, it never(?) achieved market dominance, but it did ultimately make a huge dent. And a big part of that was because AMD x86 was pretty close and pretty compatible with Intel x86 at a fraction of the cost.

Re: GLM 5.2 and the coming AI margin collapse

#32

Post by throwdbaaway »

stingraycharles wrote: post #27
throwdbaaway wrote: post #9

Seems like a pretty pointless post that still centers around output tokens. In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.

> MLA/CSA/HCA Aren’t these techniques all “lossy” compression, and one of the reasons people complain about loss in quality as the context size grows larger?

Indeed they are all lossy. Not sure how much they contribute to the quality loss in long context though. I got a 700k session with DSV4 Pro (official API), and the model was still coherent and didn't make any tool call error.

Re: GLM 5.2 and the coming AI margin collapse

#33

Post by coldtea »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

GitHub, Slack, and Office have network effects and transition costs.

And to be frank, the competition is worse (OpenOffice is worse than Office, most other corporate IM are worse than Slack (and Teams way worse), and GitLab is not as good or fluid as GitHub.

Re: GLM 5.2 and the coming AI margin collapse

#35

Post by tarpitt »

budsniffer952 wrote: post #4

> the least understood upcoming shift in AI economics. Then proceeds to talk about something in the AI news every day. Hey, did you guys hear? Open source models are cheaper and their quality is increasing! So, first, by no measure is GLM5.2 as good as Opus. Second, yes, open source models will put pressure on margins...eventually. Everyone knows that. But do you think today's AI business model is the same as tomorro…

GLM-5.2 is not as good as Opus, it's better. I can abliterate GLM-5.2 and have it work on projects that Opus refuses.

Re: GLM 5.2 and the coming AI margin collapse

#36

Post by _pdp_ »

IMHO, cheaper inference means higher costs overall :) because everyone will use more thus driving up the investment required to stay current or to compete.

Switching models is also kind of easy but not plug-and-play. Most harnesses out there do very poor job with the open weight models. Unlike Opus, GLM 5.2 ends up in loops and hallucinates a lot more. If your harness is built on the expectation that the LLM will perform well, then switching to GLM 5.2 will be an uphill struggle. We had to refactor our harness and introduce more defences because of GLM.

The cost savings are substantial. Obviously it really depends on your workloads but it is noticeable cheaper for agentic work. Coding - I don't know. We do have some coding agents on GLM 5.2 and what I noticed with some landing page experiments that the results between GLM and Opus are identical - they might be using the same training data? Obviously Opus is still substantially better model. I don't think there is an argument to be made here but GLM 5.2 is cost effective and really good too.

Overall, we switched all of our internal agents to GLM 5.2 and because it is Open Weight we are in talks to get the model from certain geo locations giving us more freedom as well as extra protection.

Overall I think this industry will be in much better place because of GLM 5.2 and whatever open-weight models come next.

Re: GLM 5.2 and the coming AI margin collapse

#37

Post by lompad »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

They don't just need healthy margins, they need to make back almost a trillion dollars in a couple of years. Comparing that to elastic search and redis doesn't make much sense.

Hyperscalers work because it actually has value compared to free offerings and because of the absolutely massive cost of switching providers.

Similar with Windows and macOS. Extremely high cost of switching to something different, if possible at all.

Same with office. Extremely high cost of switching due to compatibility issues and retraining of staff.

Your post primarily shows: It's all about lock-in. So far, it doesn't look like LLMs have any of that. So I don't think your points are valid here at all.

Re: GLM 5.2 and the coming AI margin collapse

#39

Post by Applejinx »

felixfurtak wrote: post #18

> It turns out that nearly every agentic session does a lot of web searching for looking up items This is why Google will win the race over most of its competitors. They own search.

If they did I wouldn't have had to go to DDG. It's not like it's a big jump over what used to be. I left claw-marks in Google Search, if they drove me off they're in trouble, because I didn't want to accept reality for quite some time.

Re: GLM 5.2 and the coming AI margin collapse

#40

Post by cma »

_pdp_ wrote: post #36

IMHO, cheaper inference means higher costs overall :) because everyone will use more thus driving up the investment required to stay current or to compete. Switching models is also kind of easy but not plug-and-play. Most harnesses out there do very poor job with the open weight models. Unlike Opus, GLM 5.2 ends up in loops and hallucinates a lot more. If your harness is built on the expectation that the LLM will per…

Are you running unquantized GLM-5.2 and getting in loops or quantized?
Post reply on HN