Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 and the coming AI margin collapse

martinalderson.com

Post reply on HN
Print view View original

321–330 of 495 posts

Re: GLM 5.2 and the coming AI margin collapse

#321

Post by mewpmewp2 »

AussieWog93 wrote: post #84
cogman10 wrote: post #31

Earlier quoted context omitted.

Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them. The UX is the same regardless the provider. You send in a prompt, it spits back an answer. In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with.…

>switching out an LLM is both cheap an easy. Honestly, these days probably less friction switching out Redis or Elasticsearch (backend) than changing LLM provider (human facing). Fable is seriously good enough now to, in a 20k line project, take "replace Mongoengine with raw PyMongo" and not screw anything up.

How could switching out a stateful service be easier than stateless?

Re: GLM 5.2 and the coming AI margin collapse

#322

Post by emsimot »

reinitctxoffset wrote: post #306
bushido wrote: post #295

Earlier quoted context omitted.

It seems like you're basing your spend on the subsidized consumer subscriptions. The equivalent API costs for these subscriptions is usually 12-20x. Any overages (hourly/weekly/model) on these plans gets billed at rack API costs. Its not practical to expect these subsidies to last for very long.

Agentic workloads are the most batch friendly, latency insensitive, geography insensitive, migration insensitive tokens that a big lab ever sells. In the ads business such inventory is called "remnant". The sausage is made of whatever is left over when the choice cuts have been removed. This talking point from Anthropic that Claude Code sitting in a Ralph Loop is burning top sirloin interactive session tokens is bad…

What are the choice cuts?

Re: GLM 5.2 and the coming AI margin collapse

#323

Post by breckenedge »

AmazingTurtle wrote: post #246
dgellow wrote: post #208

Earlier quoted context omitted.

> I have no idea what people are doing to burn so many tokens Agentic workflows is what consumes a lot. When you have an automated agentic loop working towards a given goal. If you use an LLM as a support for your own work you don’t end up consuming that much tokens, if you have multiple agents working on things independently, reviewing the work of other agents, etc you very, very quickly burn all your budget

I learned that running goal for hours produces exponentially more slop than running targeted prompts over and over again manually. Personally, I use gpt 5.5 high with planning every time and plan various smaller features/changes in parallel, then approve them one after another. This allows me to steer it (which I need more often than not) before approving the plan, thus reducing the otherwise accumulating slop. Using…

Like a junior engineer, I find the models to be too ambitious and unable to steer themselves at a high level yet. What I’ve done to address this is prompting the model to break down its plans into more atomic steps. For whatever reason, they’re still lazy at planning.

Re: GLM 5.2 and the coming AI margin collapse

#324

Post by poisonborz »

throwawayffffas wrote: post #275
01100011 wrote: post #151

I'll agree but from the other direction. AI continues to absorb my job as a senior systems software engineer (c/c++) and after a couple months I've only spent a few hundred dollars using gpt-5.5/5.6 and codex. I have no idea what people are doing to burn so many tokens but for me this is laughably cheap and every day I discover new capabilities. I don't care if costs go up or down, it's so cheap for what I get that I…

Asking claude to implement a single feature that takes under 30 minutes consumes 10-30 dollars of tokens in api costs.

Why would you do that? Why not use it to create a detailed plan and let it implement by a low cost model?

Re: GLM 5.2 and the coming AI margin collapse

#325

Post by galangalalgol »

ak1ng wrote: post #317
nostrademons wrote: post #115

Earlier quoted context omitted.

There's a huge case of survivorship bias when trying to recall historical analogues, because in every instance where margins collapsed and competition made the industry a commodity business, the big proprietary names are no longer with us. Here's a selection of examples, though: 1. Memory chip margins collapsed so much in the 80s that Intel exited the memory chip business entirely. At the time, they were known much m…

Sun didn't die because of the workstation market. It survived much longer.

It pivoted to server market and people who didn't think running linux was professional enough. Workstations were being held afloat by that pivot. Also they were general purpose instead if limiting their workstation market to just one niche.

Re: GLM 5.2 and the coming AI margin collapse

#326

Post by pocksuppet »

Dumblydorr wrote: post #278
nostrademons wrote: post #157

Earlier quoted context omitted.

Somewhere else in the comments here, someone else remarked "Individuals perhaps [move to the new models], but not organizations." That's illustrative. The mechanism by which organizations are forced to update their technology, move to more competitive suppliers, and cut costs is a recession. In one, every business that doesn't do so goes bankrupt, and what's left are the more efficient businesses that have adopted te…

“Brought on by government edict” No, brought on by a novel pathogen that killed 10 million people. It would’ve been much worse without government action.

Without the indecisive government action that was taken, millions more people would have died, but the economy would have done better.

With decisive government action (see New Zealand), millions less people would have died, and the economy would have done better.

Re: GLM 5.2 and the coming AI margin collapse

#327

Post by CuriouslyC »

twelve40 wrote: post #184
intrasight wrote: post #51

Earlier quoted context omitted.

You left out the one that will: federal government industrial policy

So the federal government industrial policy is the thing that supposedly will keep the prices on "A and O" high in the US while the rest of the world will get comparable AI competing to get cheaper and cheaper?

Basically, the US govt will say that foreign models and providers are a security risk and ban them. If the US has shares of Anthropic/OAI due to a sovereign wealth fund, it'll be billed as domestic industry protectionism too.

Re: GLM 5.2 and the coming AI margin collapse

#328

Post by mjhay »

saberience wrote: post #269
peab wrote: post #107

Earlier quoted context omitted.

It really depends on what you're doing, but most LLM usage and agentic runs are pretty interchangeable in my experience, and it's usually trivial to switch. If anything, you're better off supporting multiple LLMs as backup because most model providers have been so inconsistent with working all the time

Dude it’s not trivial to switch because the behaviors are different! You’re clearly not building a product based on an LLM. I’m still using various old Anthropic and OpenAI models for products I’ve built and released because I can’t risk the behavior changing in unpredictable ways and the users being pissed. It’s much easier to switch out some deterministic software than an LLM which you’ve spent a ton of time on tes…

The behavior of a single model and version can and does change. There’s not only built-in stochasticity, but closed hosted models like Claude are tweaked and changed all the time.

Re: GLM 5.2 and the coming AI margin collapse

#329

Post by rando77 »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

People might not want to be on update treadmill of proprietary weight models, for enterprise things. Things change a lot between models and they can't guarantee backwards compatibility like you can for deterministic software

Re: GLM 5.2 and the coming AI margin collapse

#330

Post by xienze »

peepee1982 wrote: post #201

There is mention of GLM 5.2's poor web search capabilities, but I see that as a harness responsibility. I've set up my own SearXNG instance on my VPS and integrated it into Pi alongside the webfetch tool, and GLM 5.2 has so far been great at finding things. I asked it to give me the current news from an Austrian online newspaper that's difficult to parse because of its aggressive ad overlays. Both ChatGPT and Claude…

[dead]
Post reply on HN