Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Companies are scrambling to curtail soaring AI costs

economist.com

Post reply on HN
Print view View original

11–20 of 31 posts

Re: Companies are scrambling to curtail soaring AI costs

#11

Post by dbuxton »

andy99 wrote: post #7

In some sense, devs are now responsible for a P&L the way a business manager would be, but I suspect that nobody is paying enough attention to the P part of it. When you incentivize people to spend as much money as possible but don’t hold them accountable properly for what they actually produce, this is what happens.

It is just extremely difficult in most companies to draw a line between a specific feature and an amount of revenue. A lot of engineering has a stochastic type of impact.

(It’s often easier to see what a feature does to reduce costs though)

Re: Companies are scrambling to curtail soaring AI costs

#12

Post by dbuxton »

motza wrote: post #6

I work at a very big company and I have unlimited access to Codex with no visibility on token usage. I don't know if this is the norm but seems kind of crazy to everyone in my team

At the moment a lot of companies thinking of it like speculative capex - we can just stop doing it if we don’t get results, or we can easily optimize.

My concern is that with all the bundling that the model labs are doing the lock-in becomes harder than anticipated to unwind

Re: Companies are scrambling to curtail soaring AI costs

#13

Post by ValentineC »

8cvor6j844qw_d6 wrote: post #4

For those that recently switched from Claude, does Codex gives you more usage than Claude Code on the highest personal plan? I frequently max out my weekly usage, and given this [1], hopefully Codex might give me more milage. [1]: https://news.ycombinator.com/item?id=48126429

I have both the Claude and ChatGPT entry-level business plans, and I feel like I get more use out of Claude, and that's even with some Fable and Opus use.

Re: Companies are scrambling to curtail soaring AI costs

#14

Post by m101 »

8cvor6j844qw_d6 wrote: post #4

For those that recently switched from Claude, does Codex gives you more usage than Claude Code on the highest personal plan? I frequently max out my weekly usage, and given this [1], hopefully Codex might give me more milage. [1]: https://news.ycombinator.com/item?id=48126429

Yes, according to this:

https://x.com/SemiAnalysis_/status/2064815044085318040?s=20

also, codex apparently uses fewer tokens to complete tasks, in part because their tokeniser is about 80% the length of that in tokens for the same string as that of the latest anthropic ones. This might be fable only though.

Re: Companies are scrambling to curtail soaring AI costs

#15

Post by rwmj »

andy99 wrote: post #7

In some sense, devs are now responsible for a P&L the way a business manager would be, but I suspect that nobody is paying enough attention to the P part of it. When you incentivize people to spend as much money as possible but don’t hold them accountable properly for what they actually produce, this is what happens.

I wonder how it's possible for a developer to assign profit. The article mentions Uber's $1500 limit per developer per month. At work we're using an LLM to analyze Windows crash dumps, which turns out to be quite expensive -- several dollars per dump, and you might analyze many every hour. Others don't use AIs very much. Should those not using so many tokens donate them to the crash dump people? And back to your point, how can we assign a profit to this? Customers love having their crash dumps analyzed quickly, but that's not the same as it being profitable.

Re: Companies are scrambling to curtail soaring AI costs

#16

Post by small_model »

motza wrote: post #6

I work at a very big company and I have unlimited access to Codex with no visibility on token usage. I don't know if this is the norm but seems kind of crazy to everyone in my team

Same, however managers are starting to ask questions, like 'what can we do to help remove bottlenecks' (they think sprinkle unlimited tokens -> revenue multiplies). In actual fact what happens is devs benefit.

It's like the company gave every senior dev a super eager brilliant fellow coder to do their work for them so they can 'do more important things' i.e. Netflix and sleep.

'Yes still working on this ticket' When codex/cc completed it flawlessly last week. The upside is absorbed by the devs knowing they could be soon surplus to requirements.

Re: Companies are scrambling to curtail soaring AI costs

#18

Post by alfiedotwtf »

noosphr wrote: post #2

If I may suggest people move away from harasses and start using Emacs. Gptel is amazing and the rest of Emacs plays incredibly well with llm development. Since the human is always in the loop token costs are also tiny compared to anything else.

I looked at gptel and eca about a year and a bit ago… how far have they come, at the time they were more geared toward code completion and essentially a buffer for communications - but all while not having some kind of Ralph Loop, Skills, and auto-compaction.

Is this all built in these days so that you can Superpowers a prompt to the point where it will automagically chug along for hours until it’s satisfied it’s completed the end goal?

Re: Companies are scrambling to curtail soaring AI costs

#19

Post by danaris »

jstummbillig wrote: post #5

I suspect this will turn out to be a super overblown issue: AI spend is literally the easiest spend to regulate in the entirety of businesses. No machinery is grinding to a halt over it, no asset that had to be bought and is now useless. You don't even have to employ or fire staff to give it a go (of course, you can still do both for other reasons). There are a lot of options that you can try out and substitute for e…

And, if you try it out and find that for your particular circumstances there's no level of spend on AI that's giving you positive ROI, you can just...not do it.

It seems like execs these days think that if they're not doing the One Thing Everyone's Doing, they're necessarily missing out, leaving potential revenue on the table, what have you. But the only reason to use AI in a business is if it makes you more money, so if you try it, and it's not doing that...I guess the question is, who are you going to believe, Sam Altman or your lying eyes?

Post reply on HN