Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's Affordability Crisis

blog.dshr.org

Post reply on HN
Print view View original

191–200 of 436 posts

Re: AI's Affordability Crisis

#191

Post by guax »

SpicyLemonZest wrote: post #85
rhinoceraptor wrote: post #62

Earlier quoted context omitted.

How are Anthropic and OpenAI going to compete on price when they're both already deeply unprofitable?

They may not be able to! It's pretty widely acknowledged, for example, that if there's some surprising plateau hiding around the corner they're both going to fail. But that could mean that they're over charging for AI usage to get research money and sustainable rates are lower rather than higher.

I think that for coding we're past the plateau issue. The frontier models of today are good enough and very valuable. The expensiveness in running them will eventually be solved by cheaper faster hardware.

I do hope that a day will come where you can buy the nvidia spark thingy for 5k that can run the equivalent of Opus 4.6 or 4.5 locally and that would be a massive thing.

Re: AI's Affordability Crisis

#193

Post by caconym_ »

deweywsu wrote: post #124

I know a lot of level-headed engineers here may not side with me, but I say let the companies who abandoned their people at the drop of a hat, with CEOs who waved their flag around on social media, proudly declaring how they'd now run their companies with 75% fewer employees wither and die. If I had been let go, there's no way I'd go back to a company like that, and there should be a black list of CEOs who acted this…

I'll believe it when I see it, but I would love to see it.

Re: AI's Affordability Crisis

#194

Post by ofjcihen »

steveBK123 wrote: post #44

I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…

I do a lot of client work for fortune 100’s.

Over the last month I have seen companies scrambling to measure deliverables against cost. Most of the back room talk is to the affect of giving devs a small allowance ($500 a month) and then making them prove their own productivity increases (again, based on deliverables, not LoC) before they either take it away or give them more.

Obviously this won’t be on an individual basis but some kind of unit.

Either way, with how much I see these companies cutting back I have no idea how the big AI companies are going to be profitable.

Re: AI's Affordability Crisis

#195

Post by andrekandre »

827a wrote: post #71

> Zitron's numbers don't tell us the real cost of generating tokens but, subject to the assumption that the platforms are not subsidizing the token price, that means Anthropic is subsidizing their enterprise customers by up to 40 times, and OpenAI up to 70 times Neither Anthropic nor OpenAI are subsidizing enterprise customers. Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high va…

  > Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high value $200/mo plan.
they may not "allow" it, but i've seen first hand enterprises encourage employees to use these accounts personally and get reimbursed later to avoid pay-as-you-go w/limits pricing for users who do tokenmaxing as a cost control measure...

Re: AI's Affordability Crisis

#196

Post by johnvanommen »

cmiles8 wrote: post #122
jcgrillo wrote: post #84

Earlier quoted context omitted.

Assuming the analysis is right, and most (or all) of these AI companies will default on their debts, what consequences might that have?

If that happens the AI companies will first try to negotiate with their creditors and after that likely declare bankruptcy with the creditors taking over what’s left of the assets. Shareholders will be wiped out and employees will be left with nothing. Various franken-companies will emerge from the bankruptcy ashes and the world will move on with AI sans the present irrational exuberance.

> If that happens the AI companies will first try to negotiate with their creditors and after that likely declare bankruptcy with the creditors taking over what’s left of the assets.

Due to the fact that we’ve already done this before (Enron, Global Crossing) -

I’m willing to bet that there are contracts in place ALREADY, that define what happens in the event of a default.

In particular, I’ll bet that the buildings, the GPUs, the patents, etc…

All of these have probably been accounted for.

I worked at a data center that closed during the WorldCom era, and when they put the padlocks on the door, there were still websites “hosted” from the building.

I don’t know if they killed the power or what. I’d cleared out my desk long before they locked it all up. I wouldn’t be surprised to learn that these websites couldn’t get their own servers, since ownership was tied up in the courts.

In the Bay Area during that time, there were row upon row of empty office buildings.

Re: AI's Affordability Crisis

#197

Post by qnleigh »

The estimate that AI companies need to replace 27% of jobs to service their debt is interesting. But at least Anthropic and Meta seem to have their eyes on replacing software engineers.

There are ~1.6M software engineers on the US [0], earning a bit under 150k/year on average [1]. If AI companies captured all of that spend, that amounts to about 250B/year. The article assumed that they need around 300B/year to keep up with their debt.

At least based on Meta's recent behavior, forcing 30-50% of developers to switch to data labeling, it looks like that is actually their game plan.

[0] https://en.wikipedia.org/wiki/Software_engineering_demograph...

[1] https://www.indeed.com/career/software-engineer/salaries

Re: AI's Affordability Crisis

#199

Post by johnvanommen »

guax wrote: post #191
SpicyLemonZest wrote: post #85

Earlier quoted context omitted.

They may not be able to! It's pretty widely acknowledged, for example, that if there's some surprising plateau hiding around the corner they're both going to fail. But that could mean that they're over charging for AI usage to get research money and sustainable rates are lower rather than higher.

I think that for coding we're past the plateau issue. The frontier models of today are good enough and very valuable. The expensiveness in running them will eventually be solved by cheaper faster hardware. I do hope that a day will come where you can buy the nvidia spark thingy for 5k that can run the equivalent of Opus 4.6 or 4.5 locally and that would be a massive thing.

> The expensiveness in running them will eventually be solved by cheaper faster hardware.

How?

* Moores Law is almost over. The 5090 improves over the 4090 mostly because of quant improvements.

* even if the hardware improves, there’s a huge incentive to slow roll the next generation. Nobody wants to end up like Sun Microsystems. Sun’s used hardware was faster than its new hardware, once you considered price. Sun ended up competing with its own used equipment.

The most obvious place for improvement is RAM, network and storage.

If someone can bring more RAM onto the market, that will unstick things.

Re: AI's Affordability Crisis

#200

Post by burningChrome »

sdesol wrote: post #183
burningChrome wrote: post #176

Earlier quoted context omitted.

I can give you some additional anecdotal evidence to support your comment. I work at a Fortune 200 company. At first, it was the Wild West. Need an LLM? You got it. Need to or want to build an army of agents? Done and done. We literally had everything at the tips of fingers for about 3 months. Teams were building their own internal tools, the team I work on canceled contracts with several software vendors because tea…

Was there at least performance gains to be measured?

AFAIK nobody was collecting analytics. The one team I was working on had put out a goal of "30% more efficient" using AI tools. Its about as subject as you can get. We never got around to what exactly that meant before everything got shut down.

Myself and several other devs were laughing about the whole thing. The company was so amped about what AI could do they never even bothered collecting any analytics that would affirm or deny any of this had a positive impact. Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks.

Post reply on HN