Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 and the coming AI margin collapse

martinalderson.com

Post reply on HN
Print view View original

71–80 of 495 posts

Re: GLM 5.2 and the coming AI margin collapse

#71

Post by yogthos »

gnarbarian wrote: post #66

the economics of this are a little counterintuitive. is there a market saturation point for intelligence? how about for software? it seems like the more you have the more you want because you're trying to do more things. as the models get smarter I get busier because I'm doing more things...

There's definitely a saturation point depending on the complexity of the problem you're solving. For example, any model can write a small shell script to resize a video with ffmpeg for you right now, so it doesn't matter whether you're using a local Qwen model, GLM, or Fable. They'll all do a roughly comparable job and you'll end up with a working script that does what you need.

Then you have things like CRUD apps, where a model needs to write some SQL, make a service endpoint, serialize some JSON, etc. Here a local model might have a bit more trouble juggling all the pieces, but any hosted model will do just fine. If your day to day job involves working on CRUD apps, then it's basically a solved problem now.

The cases where frontier models matter are when you're solving genuinely complex problems, but that's not what most people are doing day to day. So, paying an order of magnitude for a model that has capabilities to solve problems outside the range of problems you actually work on becomes a waste of money.

There's going to be a market for these models from people who really do work on complex things on regular basis, but the question is how big that market is. Additionally, open models keep getting better, and GLM 6 or DeepSeek v5 could end up being another big jump in capability where they fully close the gap with Fable. At that point, even more of the market becomes covered by these models leaving truly complex cases on the frontier.

Another thing to consider is that most big problems can be broken down into smaller ones. That's the basis for how programming languages are structured. We have primitives which are arranged into functions, that get bundled into classes or namespaces, and so on. So, you don't need an infinitely capable model to solve big problems. You just need to be able to break large problems into smaller ones, and a model that's smart enough to decompose a problem to the point where it becomes tractable.

Re: GLM 5.2 and the coming AI margin collapse

#72

Post by blourvim »

dbalatero wrote: post #65

> Of course, this was a hugely poor read of where the costs actually lie in AI. Training - while no doubt capex intensive - is a fixed, up-front cost. You spend hundreds of millions to train a model, then you are "done". I don't understand this point that people make. If you're consistently needing[0] to train new models and the cost of training relative to the % improvement seems to go higher, isn't this just a cons…

These models rely on knowledge that are embedded in their weights, if a new library is released, a new linux version comes out, some new protocol succeeds the previous one, you want your llm to know about it. Sure you can just add that into the context window, but that has its own problems.

Unless new research, there are a few which look promising, gives a new method, training is going to be a constant cost sink.

On top of this, if you stop training, it is 6 months until someone releases an open weights model and now you are competing to give the lowest price for the same product.

Also we can't forget that this is a business that *has to* be in the global labor industry, not just a tech tool, they have to have much better models to justify the trillion dollar evaluation

Re: GLM 5.2 and the coming AI margin collapse

#73

Post by le-mark »

nozzlegear wrote: post #12
throwdbaaway wrote: post #9

Seems like a pretty pointless post that still centers around output tokens. In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.

Aren't the American AI labs desperately struggling to find a market beyond just agentic coding?

I have heard but don’t have first hand knowledge that at least one company (financial services BPO) has moved most of their previously manual processing to llms. The person I talked to wasn’t forthcoming with any detail. This is what we’d expect to see though.

Re: GLM 5.2 and the coming AI margin collapse

#74

Post by w29UiIm2Xz »

lompad wrote: post #37
fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

They don't just need healthy margins, they need to make back almost a trillion dollars in a couple of years. Comparing that to elastic search and redis doesn't make much sense. Hyperscalers work because it actually has value compared to free offerings and because of the absolutely massive cost of switching providers. Similar with Windows and macOS. Extremely high cost of switching to something different, if possible…

If AI replaces labor, that's a trillion dollars of labor. About one-fifteenth of annual labor/wage earnings.

Re: GLM 5.2 and the coming AI margin collapse

#75

Post by cmrdporcupine »

felixfurtak wrote: post #18

> It turns out that nearly every agentic session does a lot of web searching for looking up items This is why Google will win the race over most of its competitors. They own search.

Which race? As an information-providing "oracle" type model, maybe.

For practical agentic tasks? Not even close. Gemini is blatantly incompetent at tool use in an agentic harness. Even their own.

Re: GLM 5.2 and the coming AI margin collapse

#76

Post by chillfox »

ChadNauseam wrote: post #50
tarpitt wrote: post #35

Earlier quoted context omitted.

GLM-5.2 is not as good as Opus, it's better. I can abliterate GLM-5.2 and have it work on projects that Opus refuses.

I'd guess opus refusals are not an issue for 95%+ of people. Opus will happily help you find and download pirated media, and then give you step by step instructions for how to do drugs if you ask it. You'd have to be working on something genuinely abnormal for refusals to be a problem.

Like making your software secure, or worse, testing that it’s secure.

Re: GLM 5.2 and the coming AI margin collapse

#77

Post by newspaper1 »

est31 wrote: post #47
fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

Those solutions have moats: 1. the cloud moat is mostly around talent really. Try finding people who can self host the alternatives to S3 et al at the HA and the scale the businesses need. Those alternatives are usually not free either, and each product might have its creator acquired (and the product cancelled) or similar. if you're a larger business then the data lock in becomes a moat: getting your data out of the…

Interesting that LLMs remove the moats for 1 (except for data lock-in) and 3, possibly even 2 if they can convert formats on the fly.

Re: GLM 5.2 and the coming AI margin collapse

#78

Post by felixfurtak »

redrix wrote: post #57
felixfurtak wrote: post #18

> It turns out that nearly every agentic session does a lot of web searching for looking up items This is why Google will win the race over most of its competitors. They own search.

I wonder if this is an alternative (and better) revenue stream vs ads for search engines: Offer a competing web search for LLMs as an alternative to Google, and charge enterprises and LLM providers for it. I know Brave do this already. Not sure about DDG (I wonder if their agreement with Bing would allow it?)

Building a good search engine is expensive. Perhaps not as expensive as AI build out.

Market share is currently Google (91%), Bing (4%), Yandex (Google can and do already monetize automated search from AI models.

Heck, if they wanted to, Google could turn off search and make you go through their AI model to get information.

Imagine that. That's how powerful they are.

Re: GLM 5.2 and the coming AI margin collapse

#79

Post by manquer »

tough wrote: post #21
copperx wrote: post #7

Earlier quoted context omitted.

Which raises the question, which are the fastest frontier models? are the enterprise hosted Anthropic models faster than what Anthropic serves? Somehow no one talks about LLM speed.

OAI has announced an upcoming 750tok/s 5.6 served through their cerebras acquisition

> cerebras acquisition

Partnership you mean?, Cerebras went public and are trading at around 45B in market cap.

While OAI could in theory cough up that kind of money, it would massively hamper their existing committed capital outlays.

Re: GLM 5.2 and the coming AI margin collapse

#80

Post by calebhwin »

cogman10 wrote: post #31
fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them. The UX is the same regardless the provider. You send in a prompt, it spits back an answer. In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with.…

Hard disagree.

Two LLMs with the same numbers on important benchmarks could have vastly different behavior in actual deployment. Not sure if as hard to switch as Excel Libre but still not "cheap and easy".

Post reply on HN