Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's Affordability Crisis

blog.dshr.org

Post reply on HN
Print view View original

311–320 of 436 posts

Re: AI's Affordability Crisis

#311

Post by VorpalWay »

mannanj wrote: post #305
VorpalWay wrote: post #295

Earlier quoted context omitted.

Probably not, at least for DRAM. The demand has historically been very variable, and building production capacity takes multiple years. Also, spare capacity is really expensive. Thus the memory manufacturers don't want to expand, betting on it being yet another temporary bubble. Also, DRAM fabs are not really usable to make compute (CPU, GPU, etc in this context) silicon. The production lines and tech have diverged s…

How did Apple figure this out? Isn't the solution to this to sort of evolve to a unified memory architecture, which wins in speed and cost anyway?

I haven't looked at apple specifically, but generally the approach taken with HBM on GPUs etc is to make multiple chiplets and connect them together very close and with very high bandwidth. The chiplets are made with different processes, optimised for each specific use case.

There are a few different variation, such as having a carrier chip below (as an interconnect), having a small PCB that everything is mounted to, or even vertically stacking chips (AMD's 3D v-cache does this on some of their CPU models).

Re: AI's Affordability Crisis

#312

Post by vjvjvjvjghv »

qurren wrote: post #289
steveBK123 wrote: post #44

I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…

What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.

Yeah. It’s weird. I just booked 2 flights to different meetings with very questionable value in the next few weeks. The whole travel costs around 3000. I would much rather spend this money on tokens but even 300 is too much per month.

Re: AI's Affordability Crisis

#313

Post by mannanj »

asdff wrote: post #244
titzer wrote: post #80

The coming AI enshittification is going to be epic. For those of us who have been on the web for more than five minutes, we can see this a mile away. If you think search ads are annoying, pre-roll YouTube ads are annoying, streaming ads are annoying, or basically ads-on-any-screen-anywhere-at-any-time are annoying, just wait until every stupid thing is powered by AI and is subtly trying to manipulate you to buy/watch…

It is already ruining forums including this one. People posting AI slop articles. Probably a good deal of commenters using AI slop to write their comments. Can't run away from it. Eventually you will have to step away entirely because you will just be interfacing mainly with AI content. I'm about fed up with forums at this point and I doubt I will be around them in the coming months at this rate of ever increasing sl…

Wasn't this the end goal in the a totalitarian state in the novels by Orwell and sort of described in the movie Ready Player One?

The gist of it as I understand it is in a society where things are fake and incredibly extractive (where a select few, bourgeoisie or rich prioritize their interests over others like we see accelerating today) they limit the forums available for people to question them and peddle their interests on the select few that remain. If you sufficiently isolate the people, it's hard for them to tell whats real and eventually they come to accept the fake narratives as truth.

In an odd way, this sort of fits in with the theories about winners writing history, and all those weird, sort of conspiracy-laden accounts of human history having these odd unexplainable gaps or stories around it. I don't know about you, I think we are simply seeing those forces of the past working at preserving their interests and using the latest technology to do it.

Re: AI's Affordability Crisis

#314

Post by simianwords »

whimsicalism wrote: post #308
simianwords wrote: post #296

Earlier quoted context omitted.

I don't get you because I'm too stupid to parse your thoughts

thanks for the productive reply

I thought you would explain what you meant but I asked Claude and I kinda understood. Interesting analysis.

Re: AI's Affordability Crisis

#315

Post by fluoridation »

fragmede wrote: post #214
johnvanommen wrote: post #186

Earlier quoted context omitted.

> The big thing is, the western world has moved so much of the manufacturing to China I built my career on Solaris and it got rugpulled by Linux. That wasn’t because of software, it was because of hardware. Linux’s cost advantage existed because Sun hardware had huge margins, because their software was basically free. AI will probably be a repeat of this. Whoever can come up with the hardware solution that minimizes…

but not all tokens are equal and vertical integration is the name of the game. Solaris did not lose to Linux, it lost to the LAMP stack on commodity x86 hardware. without the "AMP" part, Linux would've been dead in the water.

The L part of LAMP is kinda irrelevant at the application level, though. From the point of view of running the software, SAMP or LAMP doesn't make any difference. So yes, Solaris lost the competition with Linux to be the first letter of the acronym.

Re: AI's Affordability Crisis

#316

Post by vjvjvjvjghv »

asdff wrote: post #243
KolibriFly wrote: post #233

Earlier quoted context omitted.

I feel like this is way too binary. I don't have to write every line of code myself to understand the system. I don't write my own compiler HTTP stack or database either It's more about the level of abstraction. If AI handles 80% of the grunt work and I spend my time on architecture and reviews that's still a win

This works for you because you were trained in The Old Way. Consider the people younger than you. Who are literally shutting their brain off so AI can cheat on their essays and exams. They aren't going to be good architects or code reviewers.

That’s a problem. I work quite a bit with AI now by if I didn’t have a ton of experience writing code myself things would go off the rails quickly. I often have to steer the AI in a different direction from the initial plan it comes up with.

Re: AI's Affordability Crisis

#317

Post by matheusmoreira »

bayarearefugee wrote: post #82
827a wrote: post #71

> Zitron's numbers don't tell us the real cost of generating tokens but, subject to the assumption that the platforms are not subsidizing the token price, that means Anthropic is subsidizing their enterprise customers by up to 40 times, and OpenAI up to 70 times Neither Anthropic nor OpenAI are subsidizing enterprise customers. Neither Anthropic nor OpenAI allow Business nor Enterprise customers access to the high va…

> it may underestimate the inference margins of Ant/OAI's API pricing. If true then why are neither Anthropic or OpenAI dropping their API pricing to gain market share when both are clearly doing all sorts of political and PR maneuvering to compete in a cutthroat market? Since they aren't dropping the API usage prices (and are in fact raising them in a lot of subtle ways) then one of these options almost has to be tr…

Nobody cares about their training costs. Their collapse is the optimal outcome for humanity. Oligarchs blowing trillions on a godlike AI, only for the model to leak so that everyone with the hardware can use it, is literally the best case scenario.

Re: AI's Affordability Crisis

#318

Post by bluefirebrand »

mkozlows wrote: post #302
qurren wrote: post #289

Earlier quoted context omitted.

What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.

10% of a developer's cost is something like $4000/month. Many companies are complaining at a point that's well, well below that. (I think they are being irrational, and that the mental model they have of AI costs -- "how much are we spending on tooling for this developer?" -- is going to shift over time to something more sensible, but those kinds of short-sighted companies are the ones that are having cost panics.)

> 10% of a developer's cost is something like $4000/month

What company pays developers $40000/month and are they hiring?

Re: AI's Affordability Crisis

#320

Post by 9cb14c1ec0 »

gizzlon wrote: post #140

> Sales and Marketing: $5.73 billion .. That is, OpenAI spent 44% of their revenue on sales and marketing! Anyone know what they are spending this on? Can't remember seeing one OpenAI ad.. Is it just pr and influencers? Ads in the US?

If it actually was spent on ads, it seems to me OpenAI would have to be one of the single biggest ad buyers in the world. Almost certainly they are using a somewhat broad definition of sales and marketing to cram a bunch of expenses into it to make some other category look better.
Post reply on HN