Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

GLM 5.2 and the coming AI margin collapse

martinalderson.com

Post reply on HN
Print view View original

211–220 of 495 posts

Re: GLM 5.2 and the coming AI margin collapse

#211

Post by dbalatero »

rbranson wrote: post #173
blourvim wrote: post #72

Earlier quoted context omitted.

These models rely on knowledge that are embedded in their weights, if a new library is released, a new linux version comes out, some new protocol succeeds the previous one, you want your llm to know about it. Sure you can just add that into the context window, but that has its own problems. Unless new research, there are a few which look promising, gives a new method, training is going to be a constant cost sink. On…

it does not require training a model from scratch for it to be updated. the entire LLM training process is iterative. essentially each step (there are hundreds/thousands for a training run) yields a complete, usable model. tokens with updated data can be added on top essentially at any time in the future.

That's good to know, but I think maybe the bulk of the original point is still concentrated in the "we always need new models to stay relevant" piece.

Re: GLM 5.2 and the coming AI margin collapse

#212

Post by maxglute »

01100011 wrote: post #151

I'll agree but from the other direction. AI continues to absorb my job as a senior systems software engineer (c/c++) and after a couple months I've only spent a few hundred dollars using gpt-5.5/5.6 and codex. I have no idea what people are doing to burn so many tokens but for me this is laughably cheap and every day I discover new capabilities. I don't care if costs go up or down, it's so cheap for what I get that I…

Ignorance. Bad code hygiene and poor prompting. As someone who barely codes, I had a few old vibe coded projects from pre agent days when gemini ide had basically no usage limits that ballooned to multi 1000+ line files with backlog of bugs. I only stopped because dumb models starts breaking down at the point and project was serviceable for my needs. Come agentic coding and models smart enough to fix issues, but codebase is so filthy it does it wildly inefficiently. Like a few prompts would consume my 5 hour quota. Took a few days to get a decend agent.md up and refactor codebase etc and now I'm sipping tokens. I'm sure many people are still in that boat. Many of us literally don't know any best practices and can't tell agents how to behave.

In retrospect, I should have just spend a few days learning the basics, but you don't know what you don't know. And part of me can't help but feel companies aren't exactly prompting agents to be courteous when onboarding newbies because they want people like me to get hooked, and token maxxing on their end helps. I spent few $100 more than I should getting subs/tiers I didn't need, but at the time it was small $$$ for productivity gains from going from 0-1.

Re: GLM 5.2 and the coming AI margin collapse

#213

Post by kristianp »

zuzululu wrote: post #34

i would use glm 5.2 if the servers weren't in china i mean i guess my employers wouldn't know the difference but i'd like to play it safe and keep everything in america

If you look at https://openrouter.ai/z-ai/glm-5.2#providers there's about 28 providers, including z.ai and Alibaba. Most outside of China. I've never seen so many providers for a model on there before, glm 5.2 is popular.

Re: GLM 5.2 and the coming AI margin collapse

#214

Post by petesergeant »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.

Sure, but those are all things that can be trivially provided by a large inference company. In fact, I’d trust an AWS or Cerebras contract provisioning an open model before I’d trust an Anthropic or OpenAI one.

Re: GLM 5.2 and the coming AI margin collapse

#215

Post by torginus »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

The biggest difference between the cloud and AI is that an AWS server might cost 10x of what you would pay for if you bought your own, the overall expenditure is still just a small fraction of the company budget.

In contrast, even companies who spend hundreds of thousands per employee feel the AI spend right now might be too much.

Re: GLM 5.2 and the coming AI margin collapse

#216

Post by throwaway27448 »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

> Many open source office suites exist yet none compete with the ubiquity of gsuite or office

I think this is more about collaboration being hard to solve. Without collaboration gsuite/office offer nothing.

> 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time.

Mac OS is free too, just free as in beer.

> It seems that enterprises will pay top dollar for service guarantees, integration, and someone they can sue.

In the grand scheme of things, american enterprise is filthy, filthy, filthy rich. I wouldn't imagine they're the best example of rational spenders.

Re: GLM 5.2 and the coming AI margin collapse

#217

Post by throwdbaaway »

scope2093 wrote: post #204
throwdbaaway wrote: post #193

Earlier quoted context omitted.

Well I wouldn't call it a low bar, since some of the edits were quite complex. And 1M context in less than 6GB of VRAM is truly impressive, but somehow this gets way less attention than the crappy turbo quant from Google.

I'd like to understand this please. Why would the 1M context be kept in VRAM if you're using DSV4 Pro through the API? Or did you refer to different sessions?

Different sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash takes less than 6GB of VRAM. I can't run DSV4 Pro, but it should take less than 9GB of VRAM for 1M context, based on the numbers shared in https://arxiv.org/html/2606.19348v1.

Re: GLM 5.2 and the coming AI margin collapse

#218

Post by saghm »

iknowstuff wrote: post #81
cogman10 wrote: post #31

Earlier quoted context omitted.

Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them. The UX is the same regardless the provider. You send in a prompt, it spits back an answer. In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with.…

I don’t exactly see orgs lining up to switch (and train) their employees between claude desktop and codex and whatever copilot is doing. There’s probably some inertia to those harnesses/integrations on top of the llms themselves.

What would "training" even entail for that? As far as I can tell, using these tools directly is basically identical in terms of what you need to know. If you happen to have a bunch of custom configurations, maybe you need to invest some time into porting them, but it's not clear to me why you think that anyone would need to be trained if they spent months using one tool and then suddenly had t switch to the other.

Re: GLM 5.2 and the coming AI margin collapse

#219

Post by timcobb »

01100011 wrote: post #151

I'll agree but from the other direction. AI continues to absorb my job as a senior systems software engineer (c/c++) and after a couple months I've only spent a few hundred dollars using gpt-5.5/5.6 and codex. I have no idea what people are doing to burn so many tokens but for me this is laughably cheap and every day I discover new capabilities. I don't care if costs go up or down, it's so cheap for what I get that I…

Because we pay retail consumer prices (subscriptions). Those same tokens cost many thousands on enterprise billing:/

Re: GLM 5.2 and the coming AI margin collapse

#220

Post by flanked-evergl »

fny wrote: post #5

I'm not convinced raw costs matter: 1. Compute costs collapsed since the advent of Cloud and yet hyperscalers still have fat margins. 2. Many open source office suites exist yet none compete with the ubiquity of gsuite or office. GitHub, Slack are similar examples. 3. Both Windows and macOS dominate the home desktop space despite free alternatives existing for a long time. 4. Many formerly open source infrastructure…

We will keep using Claude because internal choices made by engineers and internal gatekeeping by engineers make everything else unfeasible, and going back on that would require said engineers to admit that they did something stupid, so it's not likely to happen.
Post reply on HN