95% of Companies See 'Zero Return' on $30B Generative AI Spend • Hacker News Board

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#351

Post by kin » Thu, Aug 21, 2025, 5:54 PM UTC

I'm sick and tired of seeing similar headlines and articles that reference the MIT report. It is incredible lazy journalism and borderline click bait.

The MIT report was created by project NANDA, a very pro-AI group. Read about them here: https://nanda.media.mit.edu/

The original Fortune article here: https://fortune.com/2025/08/18/mit-report-95-percent-generat... cites specifically generative AI pilot programs.

From the article: "But for 95% of companies in the dataset, generative AI implementation is falling short. The core issue? Not the quality of the AI models, but the “learning gap” for both tools and organizations. While executives often blame regulation or model performance, MIT’s research points to flawed enterprise integration."

You could have a pile of cash and 95% of companies will fail to see zero return because they don't know how to pick up the money.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#352

Post by [deleted] » Thu, Aug 21, 2025, 5:55 PM UTC

VerminOctopus1 wrote: post #185

IgorPartola wrote: post #34

What are the actual use cases that can generate revenue or at least save costs today? I can think of: 1. Generate content to create online influence. This is at this point probably way oversaturated and I think more sophisticated models will not make it better. 2. Replace junior developers with Claude Code or similar. Only sort of works. After all, you can only babysit one of these at a time no matter how senior you…

Even “only sort of works” is too generous for point #2. A dozen Claude Code agents spamming out code is… something. But it still does not replace a human, at all, even a junior one. It’s something else entirely.

[deleted]

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#353

Post by tempodox » Thu, Aug 21, 2025, 5:56 PM UTC

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

That’s an important point. Real-life use cases are not sexy. And they don’t lend themselves to overblown hype generation and “creative marketing”.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#354

Post by sillyfluke » Thu, Aug 21, 2025, 5:56 PM UTC

doorhammer wrote: post #294

sillyfluke wrote: post #257

Earlier quoted context omitted.

You also will have saved them all the cost of the AI summaries that are incorrect as well. The parent states: >Not only are the summaries better than those produced by our human agents... Now, since they have not mentioned what it took to actually verify that the AI summaries were in fact better than the human agents, I'm sceptical they did the necessary due dillengence. Why do I think this? Because I have actually t…

Again, not the OP, so I can't speak to exactly their use-case, but the vast majority of call center calls fall into really clear buckets. To give you an idea: Phonetic transcription was the "state of the art" when I was a QA analyst. It broke call transcripts apart into a stream of phonemes and when you did a search, it would similarly convert your search into a string of phonemes, then look for a match. As you can i…

Thanks for the detailed domain-specific explanation, if we assume that some whale clients of the company will end up in the call center is it not more probable that more competent human agents will be responsible for the call, whereas it's pretty much the same AI agent adressing the whale client as the regular customers in the alternative scenario?

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#355

Post by spogbiper » Thu, Aug 21, 2025, 5:58 PM UTC

wredcoll wrote: post #216

spogbiper wrote: post #186

Earlier quoted context omitted.

yes, the entire design relies on a human to check everything. basically it presents what it thinks should be done, and why. the human then agrees or does not. much work is put into streamlining this but ultimately its still human controlled

At the risk of being obvious, this seems set up for failure in the same way expecting a human to catch an automated car's mistakes is. Although I assume mistakes here probably don't matter very much.

yes, mistakes are not a huge problem. they will become evident farther down the process and they happen now with the human only system. worst case is the LLM fails and they just have to do the manual work that they are doing now

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#356

Post by Havoc » Thu, Aug 21, 2025, 5:59 PM UTC

What I found most interesting about the stat is how media spun it. At a glance they're similar...but the closer you look the more you notice writers are playing it fast & loose:

>only 5 percent of custom enterprise AI tools reach production

>95% of Enterprise AI Pilots Fail to Boost Revenues

>Why are 95% of GenAI pilot projects failing?

>95% of Companies See 'Zero Return'

>5% of integrated AI pilots are extracting millions in value

>95% of generative AI implementations in enterprise 'have no measurable impact on P&L'

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#357

Post by runarberg » Thu, Aug 21, 2025, 5:59 PM UTC

highwaylights wrote: post #28

JCM9 wrote: post #16

We are entering the “Trough of disillusionment.” These hype cycles are very predictable. GPT-5 being panned as a disappointment after endless hype may go down as GenAI’s “jump the shark” moment. It’s all fun and games until the bean counters start asking for evidence of return on investment. GenAI folks better buckle up. Bumps ahead. The smart folks are already quietly preparing for a shift to ride the next hype wave…

I wouldn’t be surprised if 95% of companies knew this was a money pit but felt obligated to burn a pile of money on it so as not to hurt the stock price.

My favorite conspiracy theory at the moment is that this is a way for the rich to literally burn the excess money to prevent it from getting back to the working classes in an effort to keep the exploitation machine running.

Now, I don’t believe this is an actual conspiracy, but rather a culture of hating the poor. The rich will jump on any endeavor—no matter how ridiculous—as long as the poor stay poor, even if they loose money in the process.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#358

Post by Imustaskforhelp » Thu, Aug 21, 2025, 6:00 PM UTC

doorhammer wrote: post #294

sillyfluke wrote: post #257

Earlier quoted context omitted.

You also will have saved them all the cost of the AI summaries that are incorrect as well. The parent states: >Not only are the summaries better than those produced by our human agents... Now, since they have not mentioned what it took to actually verify that the AI summaries were in fact better than the human agents, I'm sceptical they did the necessary due dillengence. Why do I think this? Because I have actually t…

Again, not the OP, so I can't speak to exactly their use-case, but the vast majority of call center calls fall into really clear buckets. To give you an idea: Phonetic transcription was the "state of the art" when I was a QA analyst. It broke call transcripts apart into a stream of phonemes and when you did a search, it would similarly convert your search into a string of phonemes, then look for a match. As you can i…

I genuinely don't think that the GP is actually making someone actually listen to the transcription and summary and check if the summary is wrong.

I almost have this gut feeling that its the case (I may be wrong though)

Like imagine this, if the agent could just spend 3 minutes writing a summary, why would you use AI to create a summary and then have some other person listen to the whole audio recording and check if the summary is right

like it would take an agent 3 minutes out of lets say a 1 hour long conversation / (call?)

on the other hand you have someone listen to 1 hour whole recording and then check the summary? that's now 1 hour compared to 3 minutes Nah, I don't think so.

Even if we assume that multiple agents are contacted in the same call, they can all simply write the summary of what they did and to whom they redirected and just follow that line of summaries.

And after this, I think that your summary of seeing that they are really screwing away is accurately true.

Kinda funny how the gp comment was the first thing that I saw in this post and how even I was kinda convinced that they are one of the more smarter ones integrating AI but your comment made me come to realization of them actually just screwing themselves.

Imagine the irony, that a post about how AI companies are screwing themselves by burning a lot of money and then the people using them don't get any value out of it.

And then the one on Hn that sounded like it finally made sense for them is also not making sense... and they are screwing over themselves.

The irony is just ridiculous. So funny it made me giggle

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#359

Post by f3b5 » Thu, Aug 21, 2025, 6:01 PM UTC

systemerror wrote: post #164

spogbiper wrote: post #76

Earlier quoted context omitted.

I am working on a project that uses LLM to pull certain pieces of information from semi-structured documents and then categorize/file them under the correct account. it's about 95% accurate and we haven't even begun to fine tune it. i expect it will require human in the loop checks for the foreseeable future, but even with a human approval of each item, its going to save the clerical staff hundreds of hours per year.…

The big issue with LLMs is that they’re usually right — like 90% of the time — but that last 10% is tough to fix. A 10% failure rate might sound small, but at scale, it's significant — especially when it includes false positives. You end up either having to live with some bad results, build something to automatically catch mistakes, or have a person double-check everything if you want to bring that error rate down.

Depending on the use case, a 10% failure rate can be quite acceptable. This is of course for non-critical applications, like e.g. top-of-funnel sales automation. In practice, for simple uses like labeling data at scale, I'm actually reaching 95-99% accuracy in my startup.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#360

Post by exasperaited » Thu, Aug 21, 2025, 6:02 PM UTC

lacy_tinpot wrote: post #273

exasperaited wrote: post #236

Earlier quoted context omitted.

[removed]

The real evil people here are the "artists" that have this kind of attitude. I still remember the "Digital Art isn't real Art" people, which btw still exist among your snobby "Fine Arts" crowds. I think we should actually ban all digital art platforms, no Photoshop, no special effects, all hand drawn. And I'll use some weird weaponized empathy calling out for the human soul and human creativity. What a toxic bunch.

[removed]