Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI's Affordability Crisis

blog.dshr.org

Post reply on HN
Print view View original

421–430 of 436 posts

Re: AI's Affordability Crisis

#421

Post by thisoneisreal »

skydhash wrote: post #392
thisoneisreal wrote: post #304

Earlier quoted context omitted.

I hesitate to say this because I think the AI hype is generally overdone, but I was contemplating the other day how much I would recommend my employer spend on AI tooling for me based on my salary and it's got to be in the tens of thousands of dollars per year if I'm being honest. I'm a contractor. On my most recent client, I was able to learn a completely new domain and business, design a solution and build a full s…

> I'm a contractor. On my most recent client, I was able to learn a completely new domain and business, design a solution and build a full stack POC (a complicated one with technologies I had never used before) nearly singlehandedly in less than 2 months But did you do it well? It’s not a snack, but a question based on general observation that every is praising the process, but no comment on the outcome.

I think I've done a good job straddling the delivery/maintainability line thus far. In the near future I'm likely going to have to downshift into smaller, higher quality changes and refactors, but I've been careful to keep an eye on the output and make sure the functions are reasonably scoped, documented, and understandable. I don't anticipate any issues maintaining what I've produced so far, but it could easily get out of hand as more concepts get mixed in.

Re: AI's Affordability Crisis

#422

Post by dlenski »

burningChrome wrote: post #176
steveBK123 wrote: post #44

I think the biggest problem is not necessarily the cost to develop & serve the models, but how quickly user behavior changed with token based pricing. I know a lot of people at companies where the marching orders changed on a dime end of Q1/start of Q2. These are shops that were fully on the "use AI or die (because we will fire you)" train. Now there's monitoring, reporting, alerting not just on overall cost but on "…

I can give you some additional anecdotal evidence to support your comment. I work at a Fortune 200 company. At first, it was the Wild West. Need an LLM? You got it. Need to or want to build an army of agents? Done and done. We literally had everything at the tips of fingers for about 3 months. Teams were building their own internal tools, the team I work on canceled contracts with several software vendors because tea…

I have been a major LLM skeptic and "late adopter." I had never actually used LLMs for anything at all (either in work or personal life) until about two months ago, although I had followed their development with interest and skepticism.

A couple months ago, I started using Claude for some tasks related to processing really messy data. Our company was making a big push for its use, and a co-worker gave me a fairly impressive demo of what he was able to do with this data in an hour or two.

Our company's Claude subscription defaulted to Opus 4.7 and we were encouraged to use it as much as we could.

I don't think I ever did more than 10 or 15 prompts in a day; I tend to do very detailed prompts and then spend 10-30 minutes playing around with the results and resulting code.

Today we get this:

> TL; DR: We're nearly at NNN Claude users and growing fast. Three quick asks: (1) default to Sonnet for everyday work; we have launched XXXXXXX to help with clarity, …

> …

> As we scale, let's be smart about how we work. Claude offers several models, and for everyday work — … — Sonnet is your go-to.

Re: AI's Affordability Crisis

#423

Post by 486sx33 »

i_idiot wrote: post #301
burningChrome wrote: post #176

Earlier quoted context omitted.

I can give you some additional anecdotal evidence to support your comment. I work at a Fortune 200 company. At first, it was the Wild West. Need an LLM? You got it. Need to or want to build an army of agents? Done and done. We literally had everything at the tips of fingers for about 3 months. Teams were building their own internal tools, the team I work on canceled contracts with several software vendors because tea…

Were there any managers fired for incompetence?

[dead]

Re: AI's Affordability Crisis

#424

Post by whattheheckheck »

skydhash wrote: post #419
HDThoreaun wrote: post #414

Earlier quoted context omitted.

Who cares? The business paid. Software engineers often overestimate how much businesses care about doing a job well vs quickly/cheaply

> Who cares? The future scapegoat, aka maintainer.

More agents!

Re: AI's Affordability Crisis

#425

Post by mkozlows »

Gigachad wrote: post #338
mkozlows wrote: post #300

Earlier quoted context omitted.

If your company was giving you Copilot, they were never that far on board the AI train anyway.

Github Copilot is a different product than the Windows / Office one. Horrible naming I know but it's basically just a UI and router to pick whatever model you want.

No, I know what it is. But it's not as good as Claude Code or Codex. Harnesses matter for coding agents, it's not just the model. There's a reason Microsoft had to make its developers stop using Claude Code, and it's because they all found that Copilot sucked in comparison.

Re: AI's Affordability Crisis

#426

Post by androiddrew »

whimsicalism wrote: post #283
androiddrew wrote: post #278

The over investment by VC means that yeah, they are offering all of this below market rate. It's like Enron where they have to keep the scheme going, and dumping on retail investors is the only thing they can do now. So we are going to go through a big IPO period. Everything will fall apart because VCs already extracted the growth value, and that will show up after the bag has been passed. Things will implode. What s…

When we say below market rate, what do we mean? The token economics are definitely such that they are charging more than it costs to serve these models with reasonable assumptions on param/activated size.

No, they are still in the disrupt phase of the strategy. They have enough money to operate at a significant loss. So the play is to heavily subsidize, ingrain until a business can’t function without them. Think Jack Dorsey Thanos snap where 50% of the company is let go. Then jack up prices after a couple years.

Re: AI's Affordability Crisis

#427

Post by wqaatwt »

Gigachad wrote: post #336
zhivota wrote: post #330

Earlier quoted context omitted.

I don't see why 3 years is the right number there. I'm using a 10 year old model GPU to generate tokens locally, and given the bottlenecks, a commercial model focused on RAM and transfer speeds should have a longer depreciation curve than 3 years.

Because these GPUs in data centers chew power and take up space. If in 3 years there is a new model that processes far more tokens with the same power and time the economics quickly say the hardware is cheaper to replace than to continue running. As a hobbiest at home the numbers are different and you can afford to do something inefficient.

For some time capital costs seem to be dwarfing the operating costs (or at least energy costs) when it comes to GPUs. Of course that will likely change but it might take quite a while (unless the AI bubble violently pops)

Re: AI's Affordability Crisis

#428

Post by dzonga »

qurren wrote: post #362
bluefirebrand wrote: post #318

Earlier quoted context omitted.

> 10% of a developer's cost is something like $4000/month What company pays developers $40000/month and are they hiring?

> What company pays developers $40000/month Meta, Google, Amazon, Apple, OpenAI, Anthropic, Netflix, Nvidia, to name a few

majority of software engineers are not employed at those companies but rather at banks, small / medium startups etc.

we need to start using that median instead of using salaries from the Magnificent 7.

Re: AI's Affordability Crisis

#429

Post by simianwords »

beepbooptheory wrote: post #415
simianwords wrote: post #405

Earlier quoted context omitted.

Depends on what you mean by capture technology? The internet was "captured" by FAANG. What makes OpenAI different?

No one in FAANG was selling the internet itself.

If we both agree that there will be FAANG type companies out of AI, what would they look like?

Re: AI's Affordability Crisis

#430

Post by pchristensen »

imtringued wrote: post #381
pchristensen wrote: post #339

Earlier quoted context omitted.

A company's fully loaded cost (payroll taxes, insurance, fixed company costs, infra, real estate, etc) is usually about double the employee's salary. Half of $40k is 20k/mo, which is ~$250k/hr. That's a lot but very in-bounds for software engineering salaries.

$250k/hour is half a billion USD per year. Nobody is paid that much. Based on such a basic mistake, I wouldn't trust the rest of your comment. Everything you listed feels like made up arbitrary excuses. Most of the overhead per employee is fixed, that doubling rule of thumb is assuming an employee with a salary closer to the median. At the salary you're claiming, the overhead shrinks to something like 30% on top of t…

I didn't make those numbers up - here's a widely cited source from MIT: https://nutsandbolts.mit.edu/BBJ/EmployeeCost.php

Overhead is not fixed per employee - things like taxes and benefits scale with salary, not to mention the cost of tooling (hardware, software, AI token budget, etc) tend to be higher for higher-compensated employees.

You definitely shouldn't plan your business based on my comment, but you'd be foolish to ignore it.

Post reply on HN