95% of Companies See 'Zero Return' on $30B Generative AI Spend • Hacker News Board

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#81

Post by generic92034 » Thu, Aug 21, 2025, 4:41 PM UTC

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

> It's not going to replace anyone's job.

Is it not, in the scenario you are describing? You are saying the agents are free now to do higher-value work. Why were there not enough agents before, especially if higher-value work was not done?

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#82

Post by positron26 » Thu, Aug 21, 2025, 4:41 PM UTC

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

Given that people skimp on work that is viewed as trash anyway, how were you getting value out of the summaries in the first place?

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#83

Post by throitallaway » Thu, Aug 21, 2025, 4:41 PM UTC

butlike wrote: post #77

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

How can you double-check the work? Also, what happens when the AI transcription is wrong in a way that would have terminated the employee. You can't fire a model. Finally, who cares about millions saved (while considering the above introduced risk), when trillions are on the line?

I presume they're not using these notes for anything mission or life critical, so anything less than 100% accuracy is OK.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#84

Post by Terr_ » Thu, Aug 21, 2025, 4:42 PM UTC

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

I think the biggest issue is accurately estimating the LLM failure risk, and what impacts the company is willing to tolerate in the long term. (As distinct from what the company is liable to permit through haste and ignorance.)

With LLMs the risk is particularly hard to characterize, especially when it comes to adversarial inputs.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#85

Post by hobs » Thu, Aug 21, 2025, 4:42 PM UTC

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

That is a good thing, but that's also just a training gap - I worked with tech support agents for years in gigantic settings and taking notes while you take an action is difficult to train but yields tangible results, clarity from the agent on what they are doing step by step, and builds a shared method of documenting things that importantly focuses on the important details and doesn't miss a piece which may be considered trivial by an outsider but (for instance) defines SOP for call escalation and the like.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#86

Post by dsr_ » Thu, Aug 21, 2025, 4:42 PM UTC

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

Pro-tip: don't write the summary at all until you need it for evidence. Store the call audio at 24Kb/s Opus - that's 180KB per minute. After a year or whatever, delete the oldest audio.

There, I've saved you more millions.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#87

Post by sdeframond » Thu, Aug 21, 2025, 4:42 PM UTC

md3911027514 wrote: post #50

It’s interesting how self-reports of productivity can be wrong. For example a study from METR found that developers felt that AI sped them up by 20%, but it empirically it slowed them down by 19%. https://metr.org/blog/2025-07-10-early-2025-ai-experienced-o...

An interesting study indeed. Not enough data points but still way better than anecdata and self reporting!

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#88

Post by hobs » Thu, Aug 21, 2025, 4:42 PM UTC

generic92034 wrote: post #81

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

> It's not going to replace anyone's job. Is it not, in the scenario you are describing? You are saying the agents are free now to do higher-value work. Why were there not enough agents before, especially if higher-value work was not done?

Because call centers are cost centers - nobody pays a dime more than they have to in these situations and its all commodity work.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#89

Post by dymk » Thu, Aug 21, 2025, 4:42 PM UTC

vasco wrote: post #70

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

I wonder if the human agents agree the AI summaries are better than their summaries. I was nodding as I read and then told myself "yeah but it wouldn't be able to summarize the meetings I have", so I wonder if this only works in 3rd person.

Have you tried having it summarize the meetings you have?

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#90

Post by mbStavola » Thu, Aug 21, 2025, 4:42 PM UTC

vasco wrote: post #70

jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

I wonder if the human agents agree the AI summaries are better than their summaries. I was nodding as I read and then told myself "yeah but it wouldn't be able to summarize the meetings I have", so I wonder if this only works in 3rd person.

Part of me also wonders if people may agree that its better simply because they don't actually have to do the summarization anymore. Even if it is worse by some %, that is an annoying task you are no longer responsible for; if anything goes wrong down the line, "ah the AI must've screwed up" is your way out.