Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's Affordability Crisis

blog.dshr.org

Post reply on HN
Print view View original

271–280 of 436 posts

Re: AI's Affordability Crisis

#271

Post by stingraycharles »

Supermancho wrote: post #248
woeirua wrote: post #218

It's not an affordability crisis, it's a financial crisis. The models get cheaper super fast. By this time next year Fable 5 will cost less than Sonnet does today. That's not the problem. The problem is that many companies are going to realize that they don't get any ROI from AI. Generating code faster != more profit. Most of the Fortune 500 will likely realize this and then the token budgets will come crashing down.…

> The models get cheaper super fast. By this time next year Fable 5 will cost less than Sonnet does today. I'm not sure that's something to rely on. I would be Fable 5 will be phased out and the bleeding edge will be priced up.

I’m not entirely sure, it’s my understanding that later this year a lot of compute will go online with the latest hardware that will be significantly cheaper for inference.

The problem is rather, I think, that people always want to use the latest and greatest models. And that training is super expensive.

Potentially we’ll just see less new model releases.

Re: AI's Affordability Crisis

#272

Post by gazebo2 »

stingraycharles wrote: post #266
burningChrome wrote: post #200

Earlier quoted context omitted.

AFAIK nobody was collecting analytics. The one team I was working on had put out a goal of "30% more efficient" using AI tools. Its about as subject as you can get. We never got around to what exactly that meant before everything got shut down. Myself and several other devs were laughing about the whole thing. The company was so amped about what AI could do they never even bothered collecting any analytics that would…

> Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks. What do you mean placebo effect? They thought things were created with AI while they actually weren’t?

Probably meaning C-Suite thought productivity was up because of AI, either because A) metrics showed high AI usage or more commits/LOC or B) we're mandating AI usage, why wouldn't it increase?

Re: AI's Affordability Crisis

#273

Post by energy123 »

ofjcihen wrote: post #194
steveBK123 wrote: post #44

I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…

I do a lot of client work for fortune 100’s. Over the last month I have seen companies scrambling to measure deliverables against cost. Most of the back room talk is to the affect of giving devs a small allowance ($500 a month) and then making them prove their own productivity increases (again, based on deliverables, not LoC) before they either take it away or give them more. Obviously this won’t be on an individual…

The AI companies will be profitable if they ship the goods. Make AGI and companies will pay for it.

Re: AI's Affordability Crisis

#274

Post by surgical_fire »

HDThoreaun wrote: post #246
surgical_fire wrote: post #242

Earlier quoted context omitted.

It's impossible to tell at this point. I have no idea how much of their compute is also subsidized through deals they have with the hyperscalers, etc. It's irrelevant how big their models may or may not be. Depreciation needs to be taken into account, so does actual compute expenses. Training those models is not cheap, and you will never reach a point where a model is "final". You will always need to train the next o…

Well the third party operators on open router are assuredly operating at a profit, including depreciation. The only reason they’d be profitable at 1/10 the price of the labs while the labs aren’t profitable is if inference costs the labs 10x as much per token

> Well the third party operators on open router are assuredly operating at a profit, including depreciation

They don't train new models.

They have to depreciate their GPUs, which I hope they do.

Re: AI's Affordability Crisis

#275

Post by energy123 »

onion2k wrote: post #60
fny wrote: post #34

The unit economics might be just fine. We'll know more after IPO. The drug dealer analogy has a darker side to it, however. Once your dependent, they can drive up the price just because . It doesn't need to be for existential reasons.

Once your dependent, they can drive up the price just because. It doesn't need to be for existential reasons. This is the crisis point for vibe-coders. A developer can go back to writing code by hand, as horrible as that might sound. Someone who hasn't learned to code but builds with AI can't go back. They either pay or they stop. That will be an painful choice whichever way you fall.

Using one agent at a time still won't be expensive though. Price hike will kill the agent swarm stuff.

Re: AI's Affordability Crisis

#276

Post by xboxnolifes »

a34729t wrote: post #220
thewebguyd wrote: post #189

Earlier quoted context omitted.

Even without doing that the Chinese are already going to impact our labs presence everywhere else in the world. With Fable getting pulled, any model coming out of the US is now unreliable and untrusted. No one in any other country would in their right mind choose OpenAI or Anthropic for anything. The big push for regulation and export controls is only going to ensure OpenAI & Anthropic are more like the automakers. O…

I have to push back on this: China's cheap EVs and power prices are due to industrial policy on an epic scale which goes directly against the whole free market thing. I personally think industrial policy is a good thing, but you cannot have it both ways and not expect workers to get unhappy and vote against your interests when they have no more jobs.

Even within the definition of free market there exists degrees of freedom. On one side, that id expect most Americans to be familiar with, is laissez faire capitalism. Im not sure how far it goes into the other direction before it stops being considered a free market, but if the existence of government incentives on the market stops it from being a free market, the US is also not a free market.

That is to say, I believe free markets can exist along side government policy.

Re: AI's Affordability Crisis

#277

Post by verdverm »

wqaatwt wrote: post #259
verdverm wrote: post #215

Earlier quoted context omitted.

Except that's the thing, they do stock buybacks so they do not dilute existing shareholders or lower stock prices. This is the video I watched that explained the shenanigans (from the guests' perspective, not illegal, obfuscated) https://www.youtube.com/watch?v=YrJzjC4kKCY

Yeah but that’s very standard and pretty much all pre-profit tech companies do something like that when/if they can

sure, the guests say as much, their point is that it is hard to determine their real free cash flow. They estimate Meta is 80% lower and Alphabet is 2/3 lower. They give more details and quarterly perspectives

Re: AI's Affordability Crisis

#278

Post by androiddrew »

The over investment by VC means that yeah, they are offering all of this below market rate. It's like Enron where they have to keep the scheme going, and dumping on retail investors is the only thing they can do now.

So we are going to go through a big IPO period. Everything will fall apart because VCs already extracted the growth value, and that will show up after the bag has been passed. Things will implode. What survives afterwards is what we will have.

Re: AI's Affordability Crisis

#279

Post by 827a »

andrekandre wrote: post #195
827a wrote: post #71

> Zitron's numbers don't tell us the real cost of generating tokens but, subject to the assumption that the platforms are not subsidizing the token price, that means Anthropic is subsidizing their enterprise customers by up to 40 times, and OpenAI up to 70 times Neither Anthropic nor OpenAI are subsidizing enterprise customers. Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high va…

> Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high value $200/mo plan. they may not "allow" it, but i've seen first hand enterprises encourage employees to use these accounts personally and get reimbursed later to avoid pay-as-you-go w/limits pricing for users who do tokenmaxing as a cost control measure...

Yeah I just mean that if a business came to them and asked for fifty licenses to the $200/mo plan, OpenAI would tell them to kick dirt and basically pay API pricing. Startups should 100% just be telling their employees they can expense up-to $whatever/mo in AI-related expenses, and let software engineers go buy personal Codex/Claude subscriptions.

Re: AI's Affordability Crisis

#280

Post by Quarrelsome »

burningChrome wrote: post #200
sdesol wrote: post #183

Earlier quoted context omitted.

Was there at least performance gains to be measured?

AFAIK nobody was collecting analytics. The one team I was working on had put out a goal of "30% more efficient" using AI tools. Its about as subject as you can get. We never got around to what exactly that meant before everything got shut down. Myself and several other devs were laughing about the whole thing. The company was so amped about what AI could do they never even bothered collecting any analytics that would…

30% is my new favourite stat. It seems to come up a lot, like the old "this work will take 6 months" that I was equally suspicious of back in the day.
Post reply on HN