Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 and the coming AI margin collapse

martinalderson.com

Post reply on HN
Print view View original

41–50 of 495 posts

Re: GLM 5.2 and the coming AI margin collapse

#41

Post by vmg12 »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

> I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.

Intelligence has diminishing returns, the analogues are with humans. It's a waste to hire Albert Einstein for $X million to operate the cash register in a gas station.

Artificial super intelligence will not have many customers.

Re: GLM 5.2 and the coming AI margin collapse

#42

Post by markasoftware »

sailfast wrote: post #26

How long will that $4.40 rate persist? Until we know more about the real unit economics it will be damn near impossible to rely on steady inference costs or make them predictable at the enterprise level. Gonna be a wild ride for awhile.

Multiple providers (who need to make a profit) offer the same 4.40 rate for glm-5.2. It's not subsidized.

Deepseek's 0.86 or whatever is likely subsidized but alternate providers offer it for a price comparable to glm-5.2.

Re: GLM 5.2 and the coming AI margin collapse

#43

Post by therealdrag0 »

vmg12 wrote: post #41
fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

> I understand the arguments for a margin collapse, but I don't see any historical analogues. It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue. Intelligence has diminishing returns, the analogues are with humans. It's a waste to hire Albert Einstein for $X million to operate the cash register in a gas station. Artificial super intelligence will not have many…

Of course it’s not a waste to hire Albert Einstein to work in a Swiss patent office for normal wages ;)

Re: GLM 5.2 and the coming AI margin collapse

#44

Post by spyckie2 »

It’s important that none of these entities can collude to price fix. Having China be the competitor ensures that.

Basic microeconomics is still the easiest way to understand token economies. How is it not a competitive market (where profits go to zero?).

Anything A or O does to keep more margin, any competitor can copy or choose to undercut, and undercutting has the benefit of collecting training data. So what is going to stop gross profit of tokens going to zero except for collusion/price fixing?

Re: GLM 5.2 and the coming AI margin collapse

#45

Post by LoganDark »

tough wrote: post #21
copperx wrote: post #7

Earlier quoted context omitted.

Which raises the question, which are the fastest frontier models? are the enterprise hosted Anthropic models faster than what Anthropic serves? Somehow no one talks about LLM speed.

OAI has announced an upcoming 750tok/s 5.6 served through their cerebras acquisition

Yeah, Cerebras is the one with competitive speeds nowadays but they cost an absolute fortune. Also they don't host good models publicly. Good to see OpenAI leaning into them, can't wait until these speeds are available by subscription

Re: GLM 5.2 and the coming AI margin collapse

#46

Post by throwdbaaway »

throwdbaaway wrote: post #14
throwdbaaway wrote: post #9

Seems like a pretty pointless post that still centers around output tokens. In agentic coding, cached input tokens is 90% of the API "cost". It doesn't require GPU compute, and DeepSeek has shown that it can be done 50~100x cheaper with MLA/CSA/HCA, and a whole bunch of disks. This should collapse the margin.

The current top comment in https://lobste.rs/s/ua1gxl/glm_5_2_coming_ai_margin_collapse correctly zoomed into cached input tokens, but landed on the opposite conclusion: > That is, for your $100/month fee, you get $3600 equivalent of API usage. This is presumably because Anthropic has figured out some clever things to do with model routing and input caching, and also can subsidize with investor money and take a hit o…

While we are all speculating, Boris kindly provided some guidance in https://news.ycombinator.com/item?id=47880089

> The challenge is: when you let a session idle for >1 hour, when you come back to it and send a prompt, it will be a full cache miss, all N messages. We noticed that this corner case led to outsized token costs for users. In an extreme case, if you had 900k tokens in your context window, then idled for an hour, then sent a message, that would be >900k tokens written to cache all at once, which would eat up a significant % of your rate limits, especially for Pro users.

Using the current Opus pricing, that pre-lunch 900k tokens should roughly consist of:

720k input tokens = 0.72 x $5 = $3.6

180k output tokens = 0.18 x $25 = $4.5

900k 1h cached writes = 0.9 x $10 = $9

500M cached input tokens = 500 x $0.5 = $250

$267.1 in total, with 93.6% from cached input tokens. The portion that requires GPU compute is about 3% of the total.

Post-lunch, the 900k tokens should consist of:

900k input tokens = 0.9 x $5 = $4.5

900k 1h cached writes = 0.9 x $10 = $9

So Anthropic is fine with the $267.1 accumulated over 3~4 hours before lunch, but not fine with the $13.5 incurred immediately after lunch. Why?

The only plausible explanation is that the actual cost of caching is way less than the API pricing. If you use a coding plan, Anthropic doesn't really care about your cached input tokens usage. Indeed they want you to show your ccusage screenshots. On the other hand, if you pay by API tokens, the margin is huge for cached input tokens.

Only when you do something that requires a lot of FLOPs, e.g. the post-lunch 900k input tokens, the cost becomes real.

Re: GLM 5.2 and the coming AI margin collapse

#47

Post by est31 »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

Those solutions have moats:

1. the cloud moat is mostly around talent really. Try finding people who can self host the alternatives to S3 et al at the HA and the scale the businesses need. Those alternatives are usually not free either, and each product might have its creator acquired (and the product cancelled) or similar. if you're a larger business then the data lock in becomes a moat: getting your data out of the cloud is prohibitively expensive. Furthermore, large businesses have sweet discounts.

2. ms office has immense networking effects due to its formats being quasi standards in many industries. try sending an odt to a government entity. As for gsuite, it uses open formats but it's classical google fashion a large suite of software bundled together and not that expensive for what it offers.

3. Linux is not a free alternative if you're a business, you still need to pay someone to support the computers with linux on it, and operating systems have the strongest network effects ever. Linux also has no stable ABI so one can't easily deploy third party software for it.

What's the LLM moat? Codex is OSS and Claude has gazillions of alternatives. Cursor is a nice app but it's a bunch of patches on top of vscode, a team of 5 people can vibecode it in 6 months.

Re: GLM 5.2 and the coming AI margin collapse

#48

Post by zackify »

KronisLV wrote: post #3

They have a vision MCP to make up for the model itself not having the capability natively: https://docs.z.ai/devpack/mcp/vision-mcp-server I also found their web search to be mostly okay. Furthermore, in case this is of interest to anyone, if you use their ZCode harness then you get bigger Coding Plan quotas: https://zcode.z.ai/en Used it for a bit, it sits somewhere between OpenCode Desktop (still new but nice) and…

I switched to yearly Cline pass because it was too cheap haha

Re: GLM 5.2 and the coming AI margin collapse

#49

Post by HDThoreaun »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

I think the big thing here is that paying high margins on a relatively small expense is much more palatable than high margins on a big expense. If a company is spending $1 billion/yr on tokens that a really big incentive to find an alternative where spending $1 million/yr on some SaaS with even higher margins can feel like an easy choice.

Re: GLM 5.2 and the coming AI margin collapse

#50

Post by ChadNauseam »

tarpitt wrote: post #35
budsniffer952 wrote: post #4

> the least understood upcoming shift in AI economics. Then proceeds to talk about something in the AI news every day. Hey, did you guys hear? Open source models are cheaper and their quality is increasing! So, first, by no measure is GLM5.2 as good as Opus. Second, yes, open source models will put pressure on margins...eventually. Everyone knows that. But do you think today's AI business model is the same as tomorro…

GLM-5.2 is not as good as Opus, it's better. I can abliterate GLM-5.2 and have it work on projects that Opus refuses.

I'd guess opus refusals are not an issue for 95%+ of people. Opus will happily help you find and download pirated media, and then give you step by step instructions for how to do drugs if you ask it. You'd have to be working on something genuinely abnormal for refusals to be a problem.
Post reply on HN