Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's Affordability Crisis

blog.dshr.org

Post reply on HN
Print view View original

371–380 of 436 posts

Re: AI's Affordability Crisis

#371

Post by grebc »

steveBK123 wrote: post #334
stingraycharles wrote: post #266

Earlier quoted context omitted.

> Even some of my team members were talking about the placebo effect AI has had on a lot of C-Suite folks. What do you mean placebo effect? They thought things were created with AI while they actually weren’t?

For a lot of C-suite there has been more of a fast-follow approach to AI. CEO hears 3rd hand what competitors are doing, tells his CTO to "do AI" and "hire a head of AI". So a lot of motion (do AI) was created without a destination (product outcome). The motion was what was being measured (we are doing AI).

Headlines is what matters at the executive and board level. Start measuring things and they might not be able to spin a story to sell.

For a large part of society we have management in leadership positions.

Re: AI's Affordability Crisis

#372

Post by orcasushi »

I would say making the model larger and larger will soon reach the "the point of diminishing returns". Even if the model might still get 'smarter' this might not be a good thing because the consumer (us) might not understand at all what the ai is doing. Or it's intelligence will just not be of practical value.

After this it will be hardware optimizations and tooling specializations (having different models for different tasks) and running the whole model might not be too expensive anymore. Will they be outcompeted by cheaper Chinese competitors or open models? Possibly.

I hope we reach the point I can run the whole darn thing on a old laptop.

Re: AI's Affordability Crisis

#373

Post by grebc »

steveBK123 wrote: post #341
sdesol wrote: post #183

Earlier quoted context omitted.

Was there at least performance gains to be measured?

I saw incredible productivity gain call-outs from non-technical teams like "it saves me 15 minutes per day summarizing my email". Trillions of CapEx for good Clippy/Siri, nice.

The only thing the people in charge care about is this: is their equity going up? Nothing else matters like results or revenue. Quaint concept those.

Re: AI's Affordability Crisis

#374

Post by oblio »

thisoneisreal wrote: post #304
qurren wrote: post #289

Earlier quoted context omitted.

What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.

I hesitate to say this because I think the AI hype is generally overdone, but I was contemplating the other day how much I would recommend my employer spend on AI tooling for me based on my salary and it's got to be in the tens of thousands of dollars per year if I'm being honest. I'm a contractor. On my most recent client, I was able to learn a completely new domain and business, design a solution and build a full s…

For consulting, sure. Most of the work is hitting the ground running and figuring out the old tech stacks, creating PoCs and then moving them to production and them moving on to the next "engagement".

I think the problem is more for long lived projects and developers as regular employees. As you keep working on the product, you should be developing your own mental model of everything: business, tech stack, how the company approaches things. And that's where LLM value drops down a lot, or it should, otherwise there is a bigger problem with the developer. A huge LLM cost for a consultant is in my opinion a lot more justified than a huge recurring cost for someone who's been working on the same thing for at least 9+ months.

Re: AI's Affordability Crisis

#375

Post by oblio »

steveBK123 wrote: post #332
brandensilva wrote: post #307

Earlier quoted context omitted.

Yeah the cost doesn't justify the value. Spending x3 on tokens is meaningless if they aren't seeing the profit side scale with it. Going faster on product features doesn't mean it translates to more money. In fact I'm not even sure some customers can handle the speed at which a team could move with feature delivery. So scaling horizontally in different markets seems like an advantage if a product is already mature. W…

> Going faster on product features doesn't mean it translates to more money. Exactly this. Unless the new features directly drive new users or new revenue from existing users, for many products iterating 2x faster does not mean 2x ROI. Additionally, from anecdotes here, at work, and in my network.. a lot of the unlocked developer velocity is going to fun/frivolous/extra things. I think part of it is developers have t…

> a lot of the unlocked developer velocity is going to fun/frivolous/extra things

https://en.wiktionary.org/wiki/yak_shaving

And I'll have you know I'm a Grand Master Yak Shaver.

Re: AI's Affordability Crisis

#376

Post by oblio »

christophilus wrote: post #357
pchristensen wrote: post #339

Earlier quoted context omitted.

A company's fully loaded cost (payroll taxes, insurance, fixed company costs, infra, real estate, etc) is usually about double the employee's salary. Half of $40k is 20k/mo, which is ~$250k/hr. That's a lot but very in-bounds for software engineering salaries.

Hm. I’ve been way, way underselling myself this whole time.

Not necessarily. Those jobs are very hard to get and you have to live in the right places. At most 1% of software devs will make that much, at any point in their careers.

Re: AI's Affordability Crisis

#377

Post by oblio »

aryehof wrote: post #360
qurren wrote: post #289

Earlier quoted context omitted.

What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.

They may be getting 3 times the work done, but what work? Quality effective code that meets requirements? Or something else?

> Quality effective code that meets requirements...

... and that also improves the company's financials.

Otherwise it's all fluff.

Re: AI's Affordability Crisis

#378

Post by hdgvhicv »

whimsicalism wrote: post #294
qnleigh wrote: post #197

The estimate that AI companies need to replace 27% of jobs to service their debt is interesting. But at least Anthropic and Meta seem to have their eyes on replacing software engineers. There are ~1.6M software engineers on the US [0], earning a bit under 150k/year on average [1]. If AI companies captured all of that spend, that amounts to about 250B/year. The article assumed that they need around 300B/year to keep u…

obviating software engineers is effectively AGI-complete and entails obviating most labor in existence .

If that doesn’t happen then open models catch up in months.

Re: AI's Affordability Crisis

#379

Post by oblio »

energy123 wrote: post #273
ofjcihen wrote: post #194

Earlier quoted context omitted.

I do a lot of client work for fortune 100’s. Over the last month I have seen companies scrambling to measure deliverables against cost. Most of the back room talk is to the affect of giving devs a small allowance ($500 a month) and then making them prove their own productivity increases (again, based on deliverables, not LoC) before they either take it away or give them more. Obviously this won’t be on an individual…

The AI companies will be profitable if they ship the goods. Make AGI and companies will pay for it.

> Make AGI

It's easy, just:

Make economically viable cold fusion

Make flying cars

Make a perpetual motion machine

Just because AGI is 3 simple letters doesn't implicitly mean we will live to see it.

Re: AI's Affordability Crisis

#380

Post by imtringued »

qurren wrote: post #289
steveBK123 wrote: post #44

I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…

What I don't understand is why some companies are so stingy on AI token usage. If you're paying an engineer $X and they're getting 3x the amount of work done you should be happy paying up to $2X in AI tool usage. In reality many companies start complaining at their employees when they hit $0.1*X or less.

When you look at the salaries that the average HN poster brags about, that's at least ten thousand dollars worth of tokens or a billion input tokens per year. That doesn't sound like an unreasonable spending limit.
Post reply on HN