71–80 of 122 posts
Honest question: Is inference actually being sold at a loss? I.e. does inference itself cost more to run than we're being charged, or are the companies operating at a loss because they're spending more on training than the margins they get from inference?
I didn't find this very convincing. Especially the argument around artificially low cost -- we know that training the next model is the biggest cost for these companies, and we've already seen inference costs fall drastically ( https://epoch.ai/data-insights/llm-inference-price-trends ).
Yes, Dario has publicly stated that models are already profitable if you exclude R&D for the next model. Even if that’s not true, given that hardware and software efficiency gains can be expected to continue it’s likely that this is the most expensive the current level of intelligence will ever be. The frontier models may increase in price, but only because they’re also more capable. If you hold intelligence constant…
The section on "artificially low costs" does not make a lot of sense to me. If anything I feel like the costs are inflated for the frontier models, not "artificially low". Easy proof: GLM-5 costs about 1/10 as much as Opus. I'm not going to tell you it's as good as Opus 4.6 -- it's not -- but it performs comparably to where frontier models were 6 months ago. (It's on par with Sonnet 4.5 on leaderboards, though in pra…
The post is factoring in training costs, not just inference.
> in fact, they actually compound the problem by encouraging significantly more usage
because if eliminating training costs makes running the model above cost, the problem is helped by significantly more usage not compounded.
More usage compounds the problem only if inference is unprofitable.
(the article briefly mentions training but that's later).
The skill atrophy point strikes me as tenuous at best. Obviously, the plural of anecdote is not data, but I find myself able to work on projects of greater complexity than I would have been able to otherwise. 90% of my time is spent going back and forth on Markdown files, discussing the architecture, trade-offs, etc. I don't think it's necessarily impossible to use all this newfound power to ship more sloppier code.…
> while GenAI can improve worker efficiency, it can inhibit critical engagement with work and can potentially lead to long-term overreliance on the tool and diminished skill for independent problem-solving.
https://www.microsoft.com/en-us/research/wp-content/uploads/...
it's a real problem when applied to a population.
I find the other article that the author refers to in his text, to be more thorough and revealing: https://www.wheresyoured.at/the-ai-industry-is-lying-to-you/
Earlier quoted context omitted.
And even then - I still read the code it generates, and if I see a better way of doing something I just step in, write a partial solution, and then sketch out how the complete solution should work.
Unless the solution is going to be more secure, faster, more stable etc, why does it matter? Will the end user care? “Does it make the beer taste better”?
> maintainability is inversely proportional to the amount of time it takes a developer to make a change and the risk that change will break something
https://softwareengineering.stackexchange.com/a/134863
i could be wrong, but i'm pretty sure that end-users get upset when a change takes a long time or it ends up breaking something for them.
just because people are finding that agents or whatever are speeding changes up now doesn't necessarily mean they won't encounter a slow-down later when the codebase becomes an un-maintainable mess. technical debt is always a thing, even with machines doing the work (the agent/machine still has to parse a codebase to make changes).
Earlier quoted context omitted.
The post is factoring in training costs, not just inference.
No it's not. Otherwise this part doesn't make sense > in fact, they actually compound the problem by encouraging significantly more usage because if eliminating training costs makes running the model above cost, the problem is helped by significantly more usage not compounded. More usage compounds the problem only if inference is unprofitable. (the article briefly mentions training but that's later).
The skill atrophy point strikes me as tenuous at best. Obviously, the plural of anecdote is not data, but I find myself able to work on projects of greater complexity than I would have been able to otherwise. 90% of my time is spent going back and forth on Markdown files, discussing the architecture, trade-offs, etc. I don't think it's necessarily impossible to use all this newfound power to ship more sloppier code.…
> I find myself able to work on projects of greater complexity than I would have been able to otherwise Yes. Now turn off the LLM and make an improvement to that code.
But it doesn't have to be that way. You just have to put an effort into shipping fewer, better features as opposed to more features. The projects I'm working on (e.g. agent orchestration, because who isn't nowadays) have a small surface area and high payoff and thus are uniquely well positioned for this.
If I couldn't use an LLM, I would still work on this, and it would have roughly the same architecture. But because I'm able to go probably 100x as fast, I'm able to be much more ambitious. Or rather, I'm able to discover that my initial ideas were not on point and pivot, and not have any sense of sunk cost
Anyway, to each their own.
The skill atrophy point strikes me as tenuous at best. Obviously, the plural of anecdote is not data, but I find myself able to work on projects of greater complexity than I would have been able to otherwise. 90% of my time is spent going back and forth on Markdown files, discussing the architecture, trade-offs, etc. I don't think it's necessarily impossible to use all this newfound power to ship more sloppier code.…
Totally agree with this taking on projects of greater complexity. I honestly feel the sloppier code thing is going to die soon. People make mistakes too. Always see people holding the machine to like this totally different standard.
Months! That's not a long time.
Earlier quoted context omitted.
But I don't need to pay training costs to use GLM-5?
Sure, but somebody needs to pay for GLM-6 unless you're happy to stop here.