Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AI sticker shock hits corporate America

axios.com

Post reply on HN
Print view View original

61–70 of 154 posts

Re: AI sticker shock hits corporate America

#61

Post by spaceman_2020 »

stockresearcher wrote: post #43

We use GH Copilot at work and this week sat for a presentation by GH about optimizing token usage and maximizing ROI on tokens used. Anyone else get this presentation? They didn’t have time for questions because they had to run and give it to the next big enterprise on their list… They basically said that everything is too expensive, you have to watch it like a hawk. It was as if they poured a bucket of cold water on…

Just hire people and pay then a fair wage and let everyone get richer together ffs

When did American capitalism become such a zero sum game

Re: AI sticker shock hits corporate America

#62

Post by randusername »

utopiah wrote: post #38
cmiles8 wrote: post #13

Playing out in company after company right now: CEOs: “Get me some of that GenAI” CTO: “OK, we have all the GenAI” CEOs: “Employees, it’s AI or bust” Employees: Tokenmax CFO: “Um, this is costing a ton and we’re not seeing savings or efficiency materialize.” CEO: “Are we getting any value out of this?” COO: “Not really, and frankly I’m getting annoyed at all the AI slop turning up all over.” CEO: “OK, well, let’s do…

> Hopefully folks won’t blame me for the mess Lesson #1 from business school : take all credits, put the blame on others, if there no easy scapegoat blame the "economical context" Lesson #2 and beyond : just see lesson #1, it's enough. You've made it and it's ALL thanks to this amazing business school degree you got, now go profits with your new already wealthy peers.

I have a literal ledger tracking the number of times executives have used the phrase "evolving capital markets"

It's amazing to me how subtle the difference can be between a business leader that seems to know what they're doing and one that clearly does not. Same words, even, but from one mouth it's compressing a complex thought and from another it's word salad to give the appearance of complex thought.

Re: AI sticker shock hits corporate America

#64

Post by somenameforme »

imglorp wrote: post #34

This is an important inflection point. When tokens get correctly priced, all of the insane over-investment in capital will need to draw back: buying data centers, semiconductors, and politicians. Even then, it won't be right-priced with regard to actual costs. The environmental impact should have been priced in from the beginning. There seems to be a parallel with subsidizing fossil fuels, under pricing them which en…

I've yet to see any compelling data about inference being particularly expensive. For local LLM models, that are becoming increasingly viable, it's dirt cheap. The same is also true in image gen world where now even a heavily dated GPU can cheaply and quickly produce high quality images.

I also think the image gen world is a useful analog because there are a million sites, presumably still making money, with markups that are multiple orders of magnitude off their costs. They're feeding off user ignorance that was, at least in part, artificially seeded by implying high costs for image gen back in its day. Though it's possible/probable that the initial training runs were expensive, but that's a one-and-done cost.

Re: AI sticker shock hits corporate America

#65

Post by alextillman »

This is because the current AI approach relies on AI to be a glorified search engine – know everything about everything requiring enormous, ever growing models, and demanding search-engine like near instant responses requiring bigger more complex chips and sprawling data centers to run them in. This leads to a loop demanding ever bigger models, updated at a more and more expensive cost, and chipsets that become much more expensive to deploy.

If you move those things to software and utilize tools that are cheap at scale (databases, web search etc.) the hardware arms race ends and the price becomes sustainable. With the right tools preparing dynamic context for a conversation, models are used for their reasoning and not for their knowledge. And waiting even a minute or two for a model to prepare a response, evaluate it, and iterate to improve quality makes a huge difference.

Re: AI sticker shock hits corporate America

#66

Post by groestl »

fullshark wrote: post #16
6stringmerc wrote: post #7

Earlier quoted context omitted.

Counterpoint: based on a lot of anecdotes here, the most likely people to burn tokens aren’t GenZ but managers using ChatGPT to respond to questions or otherwise as an outsourcing of their job. There aren’t enough GenZ in the workforce to back your claim in my opinion. Token efficiency where instead of the AI burning money at 1:3 instead of 1:5 isn’t quite a winning argument.

There's no way managers using LLMs to answer emails are burning tokens at a comparable rate to someone trying to utilize inference in production systems is.

Maybe managers going back to coding.

Re: AI sticker shock hits corporate America

#67

Post by Esophagus4 »

throwatdem12311 wrote: post #18
voidfunc wrote: post #12

Earlier quoted context omitted.

Working conditions are fine, I simply am not incentivized to be efficient with tokens.

Yeah, everything is fine until you don’t want to use AI for something because it sucks at that task and then you end up on a PiP because your token burn is low. Why the f*ck are AI Token Use Leaderboards even a thing. Features that used to take months are now expected in days. Oh you didn’t merge 40 pull requests and deploy to prod 15 times today? Aren’t you using Opus the greatest thing since the invention of the wh…

> on a PiP because your token burn is low

Does this happen? I’ve never been at a company that measures employee performance by token burn targets. I suspect most companies don’t do that, but I could be wrong obviously.

Re: AI sticker shock hits corporate America

#68

Post by treis »

kombookcha wrote: post #14
SupLockDef wrote: post #8

Earlier quoted context omitted.

Are you saying that making a leaderboard of who is spending the most is going to be expensive? They couldn't see that coming, but for sure they can predict how the future will be when it's time to sell their "visions" of the world. Meanwhile, sheep's are going to believe and max their token usage with their own wallet. "You are so be left behind if you're not". It's a mass psychosis. The only winners here are the har…

The other day I had to read a C-suite guy share how he had an epiphany that spending more tokens did not linearly align with more useful features being output by the teams. He was describing it as this breakthrough moment for him, as if it wasn't glaringly obvious that making the KPI "spend more tokens" would result in inefficient token spending, not massive value for the customer. It's baffling how these people have…

This entire narrative is just made up. Managers know not to reward spending. At best you had some tracking to see who was using it and encouragement for those that aren't to start.

Re: AI sticker shock hits corporate America

#69

Post by 1vuio0pswjnm7 »

"“It’s a real dollar investment,” says Tan, speaking for Claudeholics who go full blast. “You actually have to spend six to seven figures on tokens—I’m on a run rate to do seven figures this year.”"

https://www.wired.com/story/how-ai-agents-plunged-tech-world...

Re: AI sticker shock hits corporate America

#70

Post by dylan604 »

lelanthran wrote: post #26

I am Jack's total lack of surprise :-/

How much longer before we get to the “I get cancer. I kill Jack” stage? Isn’t uncontrolled growth a sign of cancer? Or is it more virus like where it just continues to grow without ever reaching a balance with its surroundings?
Post reply on HN