Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's economics don't make sense

wheresyoured.at

Post reply on HN
Print view View original

111–120 of 201 posts

Re: AI's economics don't make sense

#111

Post by koliber »

xienze wrote: post #69
hparadiz wrote: post #61

Earlier quoted context omitted.

The economics is spending a few hundred bucks on software for an IC you're already paying over ten grand a month in order to make them more productive. How are supposedly smart industry experts not seeing this obvious fact? Are these guys actually experts?

> The economics is spending a few hundred bucks on software for an IC you're already paying over ten grand a month Let's be fair here, the endgame is not "a few hundred bucks a month." Not for how much money has been invested. How much extra you have to spend to make developers how much more productive, and will companies go along with it is the trillion dollar question.

A long time ago a vast majority of people on earth were farmers. They used relatively simple tools like scathes.

Over a few centuries better tools and technology made it so that <5% of the population in rich countries are farmers. They use tools like million dollar harvesters.

Re: AI's economics don't make sense

#112

Post by m4rtink »

CodingJeebus wrote: post #79
hparadiz wrote: post #61

Earlier quoted context omitted.

The economics is spending a few hundred bucks on software for an IC you're already paying over ten grand a month in order to make them more productive. How are supposedly smart industry experts not seeing this obvious fact? Are these guys actually experts?

It's a few hundred bucks per month for now, but that's not going to last. At some point, the industry is going to pivot towards tracking token-based productivity because it's not going to be cheap forever unless FOSS models catch up.

Please don't call open weight models FOSS models - that's actually very wrong, unless you actually have all the training data and can modify the data and training methodology to retrain the model yourself.

Re: AI's economics don't make sense

#114

Post by margalabargala »

GaggiX wrote: post #72
zozbot234 wrote: post #47

Earlier quoted context omitted.

It's not even a fixed cost per token (even though it's billed that way, and that's still miles better than a fixed-price all you can eat). You're incurring a cost that's proportional to generated tokens times the context for each (plus the prefill cost for any uncached input), so the expense grows quadratically with your average generated context. This all becomes extremely visible when trying to do agentic coding wi…

Using larger contexts often costs more in the APIs or consume more of your quota but this is becoming less of a problem with models using more clever attention mechanisms and not just full attention on all layers. You can look at: https://sebastianraschka.com/llm-architecture-gallery/ and see how much things have changed.

This is also something of a non issue because as context grows and attention gets diluted, the models perform worse. It'll cost Anthropic more to run your 900k context session, yes, but it's in your interest not to have a 900k session in the first place.

Re: AI's economics don't make sense

#115

Post by mrandish »

JohnMakin wrote: post #53

I've sort of lost some respect for ed that I had early on in the hype cycle - he's still right about some things, but I can see him slowly and subtly retreating from his strong position, held even a few months ago, that these things will never ever be useful for anything and it's all a scam because they don't actually do anything at all except burn money. He would say it like 8 times a monologue. I remember one podca…

I've only read a few of his pieces here and there and had just assumed he was an AI skeptic, so I never thought his position was LLMs would never be good for anything at any price. That's a pretty extreme thing for any serious person to have ever claimed. Frankly, it seems more like a straw man exaggeration of AI skepticism. I consider myself to generally be an AI skeptic, but to me that means skepticism about:

1) Nearer-term investment returns on AI businesses and data center build-outs.

2) Claims that LLMs are now (or soon will) rapidly displace most/all senior positions in certain high-skill professions (eg software engineering, music/film making, etc), leading to less overall jobs for those kinds of workers and mass unemployment.

3) The "Foom" overnight takeoff hypothesis that AI will soon be able to iteratively sustain substantial self-improvement directly yielding profound new fundamental capabilities across infinite generations with no human involvement.

I've never thought that AI isn't already quite useful for some things today, or that no investors will ever make money on AI, or that AI won't displace some workers in some types of jobs, or that using AI isn't already helping accelerate the development of AI. Just that there's been a lot of hype, exaggeration and over-estimation about how much impact, how soon and how broad. There will be a few instances of rapid, large impacts but the majority of it will be slower, more gradual and less disruptive than extreme predictions - and many of the most over-the-top predictions may not ever happen. Not because they can't happen but probably for more mundane economic, logistic and human-factors reasons along the lines of why we're no closer today to the 1950s visions of a flying car in every driveway.

Re: AI's economics don't make sense

#116

Post by margalabargala »

kgwgk wrote: post #64
asdfasgasdgasdg wrote: post #56

Earlier quoted context omitted.

>It makes me wonder if I have been living under a rock, because I have never heard of frontier labs making money. You're confusing the profit from the marginal token and overall profit (basically gross margin and operating margin). The comment you're replying to is calculating that AI labs are probably making a substantial profit per paid token. It's just that so far that profit has not been able to overcome the ongo…

> not been able to overcome the ongoing R&D and capex costs. And the cost of not-quite-paid tokens.

Which may or may not exist, hence this thread.

Re: AI's economics don't make sense

#117

Post by fancyfredbot »

He does have a point about fees. It's not really surprising that the fee structure designed for chatbots would not make sense when applied to long running tasks and agents. But an increase in prices can solve this problem.

Doubtless some people will reduce usage as a result. But Ed seems to find the idea that a 10 man developer team might spend 80K a year on tokens ridiculous. I don't understand this. Has he seen how much developers are paid? If you get a 20% productivity boost from coding agents, then that's two developers for 80K - effectively very good value.

Where things could go wrong is in comparison to cheaper models. If it's 5K a year for Qwen, and it's 2/3 as good will you pay 75K extra for Opus? Perhaps not.

Re: AI's economics don't make sense

#118

Post by margalabargala »

pmdr wrote: post #109
CamperBob2 wrote: post #74

Earlier quoted context omitted.

People who don't adjust their prior outlook in light of newer data may not be the best fit around here. I'm OK with that.

What is the newer data?

Extensively discussed elsewhere in this thread. Just start at the top and start reading comments.

Re: AI's economics don't make sense

#119

Post by kimetime »

boelboel wrote: post #100
joshjob42 wrote: post #10

There's a few major problems with the article. The most obvious is that frontier labs are not charging remotely close to the cost of tokens; afaik most estimate north of 80% profit margins. As a reference, providers are profitably providing Kimi K2.6 for $4/1Mtok out. Is that as good as Opus? No, but it's probably at least Sonnet level, so that's ~4x cheaper than Sonnet while still being profitable to serve on the ma…

Isn't this akin to saying Big Pharma companies could easily make money if they just stopped doing expensive research? The massive R&D spend is the core of the business plan; it's the only reason they can demand high prices in the first place. Once OpenAI stops spending billions on training, their pricing power vanishes because users will just migrate to Anthropic or whoever releases the next frontier model. Would imp…

Big Pharma does seem like a good comparison for frontier lab business model. Doesnt really have the patent protection or distinct diseases pharma does, Wonder if labs start more heavily branding “specialties” instead of general capabilities to develop some differentiation

Re: AI's economics don't make sense

#120

Post by legulere »

koliber wrote: post #111
xienze wrote: post #69

Earlier quoted context omitted.

> The economics is spending a few hundred bucks on software for an IC you're already paying over ten grand a month Let's be fair here, the endgame is not "a few hundred bucks a month." Not for how much money has been invested. How much extra you have to spend to make developers how much more productive, and will companies go along with it is the trillion dollar question.

A long time ago a vast majority of people on earth were farmers. They used relatively simple tools like scathes. Over a few centuries better tools and technology made it so that <5% of the population in rich countries are farmers. They use tools like million dollar harvesters.

It's not the 20x efficiency of harvesting technology compared to what agrarian societies that make them make sense. It's the productivity of the other 95% of the population that makes their labor cost so high that such expensive machines make economic sense.
Post reply on HN