GLM 5.2 and the coming AI margin collapse

Post by arcanemachiner » Tue, Jul 07, 2026, 5:06 AM UTC

tharkun__ wrote: post #99

pants2 wrote: post #86

Earlier quoted context omitted.

If you're developing on top of LLM APIs directly, this is definitely not true. There are differences in how context caching works, in what's available through native harnesses, the types of tools you're fine-tuned on (GPT uses apply_patch while Claude uses edit, with different formats), the API surface (Agents SDK, Responses API, Managed Agents), cost structures, and best-practice guidance all around. Not to mention…

Exactly, as in, really, will they? Where and at what price, especially across an actual enterprise that needs to deploy them to lots of devs? There's much more than just the actual model. Of course my numbers are a sample of one and I am not spending a lot of money or time on it. Just lazily trying things on my "happen to have this" hardware. But basically trying out the Claude Code I'm used to from work but locally…

Post by rbranson » Tue, Jul 07, 2026, 5:11 AM UTC

blourvim wrote: post #72

dbalatero wrote: post #65

> Of course, this was a hugely poor read of where the costs actually lie in AI. Training - while no doubt capex intensive - is a fixed, up-front cost. You spend hundreds of millions to train a model, then you are "done". I don't understand this point that people make. If you're consistently needing[0] to train new models and the cost of training relative to the % improvement seems to go higher, isn't this just a cons…

These models rely on knowledge that are embedded in their weights, if a new library is released, a new linux version comes out, some new protocol succeeds the previous one, you want your llm to know about it. Sure you can just add that into the context window, but that has its own problems. Unless new research, there are a few which look promising, gives a new method, training is going to be a constant cost sink. On…

Post by whs » Tue, Jul 07, 2026, 5:14 AM UTC

platinumrad wrote: post #56

est31 wrote: post #47

Earlier quoted context omitted.

Those solutions have moats: 1. the cloud moat is mostly around talent really. Try finding people who can self host the alternatives to S3 et al at the HA and the scale the businesses need. Those alternatives are usually not free either, and each product might have its creator acquired (and the product cancelled) or similar. if you're a larger business then the data lock in becomes a moat: getting your data out of the…

Linux has a very stable userspace syscall ABI. About as stable as Windows, and much more stable than MacOS or the BSDs. I agree with everything else though.

Post by KetoManx64 » Tue, Jul 07, 2026, 5:17 AM UTC

rhipitr wrote: post #146

cogman10 wrote: post #31

Earlier quoted context omitted.

Unlike all your examples, switching out an LLM is both cheap an easy. So easy that every 3 months or so new models are released and people grab them and start using them. The UX is the same regardless the provider. You send in a prompt, it spits back an answer. In all your other cases, the cost to switch is losing support and a difficult transition period. But in the case of LLMs, there was no support to begin with.…

Switching out an LLM? What do you mean by this? Sure some models can run locally but in a company with lots do people they might not be willing to spend to self host a larger model that requires beefier hardware to host, plus all the complexity to scale that out to a bit internal user-base

Post by lelanthran » Tue, Jul 07, 2026, 5:19 AM UTC

jaggederest wrote: post #110

ilaksh wrote: post #97

I think the profits depend on how well they manage their fleet purchases (or possible sub-leasing?) to get high utilization without overloading or idle racks. Because accelerators like H200, B300 etc. are highly parallel and designed to run like 200 or maybe 300 sequences at once (depends on the model, just guessing). I assume they finance the hardware and that cost per device or rack is the same whether each unit is…

Someone on HN made a comment in one of these threads that we could bake the weights into something like Cerebras's wafer scale chips and serve essentially the entire world off a single wafer, which is a pretty wild thing to think about. You'd have to make new hardware any time you trained a model but that seems really worth it.