Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Companies are scrambling to curtail soaring AI costs

economist.com

Post reply on HN
Print view View original

21–30 of 31 posts

Re: Companies are scrambling to curtail soaring AI costs

#21

Post by mittensc »

rwmj wrote: post #15
andy99 wrote: post #7

In some sense, devs are now responsible for a P&L the way a business manager would be, but I suspect that nobody is paying enough attention to the P part of it. When you incentivize people to spend as much money as possible but don’t hold them accountable properly for what they actually produce, this is what happens.

I wonder how it's possible for a developer to assign profit. The article mentions Uber's $1500 limit per developer per month. At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour. Others don't use AIs very much. Should those not using so many tokens donate them to the crash dump people? And back to your poin…

> At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour

Is that in any way useful?, how so?, are dumps from optimized builds?

I've found that most of the time I don't even need to open a dump because of regular automation providing all thread callstacks in tickets... That and logs are generally enough...

Re: Companies are scrambling to curtail soaring AI costs

#22

Post by nlpnerd »

rwmj wrote: post #15
andy99 wrote: post #7

In some sense, devs are now responsible for a P&L the way a business manager would be, but I suspect that nobody is paying enough attention to the P part of it. When you incentivize people to spend as much money as possible but don’t hold them accountable properly for what they actually produce, this is what happens.

I wonder how it's possible for a developer to assign profit. The article mentions Uber's $1500 limit per developer per month. At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour. Others don't use AIs very much. Should those not using so many tokens donate them to the crash dump people? And back to your poin…

https://seldon-ai.com/blog/how-much-of-your-llm-bill-is-just...

Re: Companies are scrambling to curtail soaring AI costs

#23

Post by sscaryterry »

jstummbillig wrote: post #5

I suspect this will turn out to be a super overblown issue: AI spend is literally the easiest spend to regulate in the entirety of businesses. No machinery is grinding to a halt over it, no asset that had to be bought and is now useless. You don't even have to employ or fire staff to give it a go (of course, you can still do both for other reasons). There are a lot of options that you can try out and substitute for e…

> no asset that had to be bought and is now useless

Dare I say the GPU bought today won't be economical in 3-4 years time?

Re: Companies are scrambling to curtail soaring AI costs

#24

Post by cyanydeez »

jstummbillig wrote: post #5

I suspect this will turn out to be a super overblown issue: AI spend is literally the easiest spend to regulate in the entirety of businesses. No machinery is grinding to a halt over it, no asset that had to be bought and is now useless. You don't even have to employ or fire staff to give it a go (of course, you can still do both for other reasons). There are a lot of options that you can try out and substitute for e…

yeah, it's easy to regulate, but the psychological impact isnt.

You spend 3 days making AI do something, and now what, you're suppose to spend 2 days trying to keep up with whatever it's done? That's like if your coworker's projects, every week, were handed to you on Thursday and you're told to finish them up.

No matter how smart you are, the bootstrap of new knowledge is impossible.

So sure, it's easy to turn the spigot off at arbitrary dollar values, but you can't just turn the worker's mindset off.

Workflow is valuable because of how in the flow it is.

Re: Companies are scrambling to curtail soaring AI costs

#26

Post by jstummbillig »

sscaryterry wrote: post #23
jstummbillig wrote: post #5

I suspect this will turn out to be a super overblown issue: AI spend is literally the easiest spend to regulate in the entirety of businesses. No machinery is grinding to a halt over it, no asset that had to be bought and is now useless. You don't even have to employ or fire staff to give it a go (of course, you can still do both for other reasons). There are a lot of options that you can try out and substitute for e…

> no asset that had to be bought and is now useless Dare I say the GPU bought today won't be economical in 3-4 years time?

How does that concern individual companies as AI consumers? The hyperscalers have no trouble utilizing their GPUs (and I would suspect they mathed it out to be profitable but, again, that's just a different topic).

Re: Companies are scrambling to curtail soaring AI costs

#27

Post by bardown59 »

Yeah, and how we build—and what we surface operationally—matters as well.

I run several AI-based products, and I’ve moved almost entirely to agentic coding, so this is a huge concern for me. One thing that has helped in the coding realm is separating planning from execution: I use a stronger, more expensive model for architecture, research, and difficult decisions, then hand the implementation work to cheaper agents where the task is more constrained.

On the product side, I treat token usage and costs like any other production resource. We record usage by request, user, feature, and model, surface it in admin dashboards, and set budgets and hard cutoffs.

Alerts are useful, but automatic limits matter more—especially as a small business guy so a 2 AM loop doesn't bankrupt me :)

Re: Companies are scrambling to curtail soaring AI costs

#28

Post by rwmj »

mittensc wrote: post #21
rwmj wrote: post #15

Earlier quoted context omitted.

I wonder how it's possible for a developer to assign profit. The article mentions Uber's $1500 limit per developer per month. At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour. Others don't use AIs very much. Should those not using so many tokens donate them to the crash dump people? And back to your poin…

> At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour Is that in any way useful?, how so?, are dumps from optimized builds? I've found that most of the time I don't even need to open a dump because of regular automation providing all thread callstacks in tickets... That and logs are generally enough...

I'm not directly involved in the day to day activity around this, but as I understand it, it's about triaging related crashes together and associating them with prior bugs and suggesting fixes, else escalating.

Re: Companies are scrambling to curtail soaring AI costs

#29

Post by mittensc »

rwmj wrote: post #28
mittensc wrote: post #21

Earlier quoted context omitted.

> At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour Is that in any way useful?, how so?, are dumps from optimized builds? I've found that most of the time I don't even need to open a dump because of regular automation providing all thread callstacks in tickets... That and logs are generally enough...

I'm not directly involved in the day to day activity around this, but as I understand it, it's about triaging related crashes together and associating them with prior bugs and suggesting fixes, else escalating.

thanks!

Associating with past bugs is a nice use case

Suggesting fixes seems risky, can lead people on wrong paths and waste time or fixing a sympthom instead of proper root case and waste the bug report?

Grouping is pretty easy without LLMs, just pattern match callstack symbols

Triage, sure, but a human would be able to do that as well in a couple of minutes for free with callstack in tickets, couple of dollars per ticket seems excessive in comparison

Post reply on HN