Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

95% of Companies See 'Zero Return' on $30B Generative AI Spend

thedailyadda.com

Post reply on HN
Print view View original

361–370 of 445 posts

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#361

Post by spogbiper »

LPisGood wrote: post #244
spogbiper wrote: post #76

Earlier quoted context omitted.

I am working on a project that uses LLM to pull certain pieces of information from semi-structured documents and then categorize/file them under the correct account. it's about 95% accurate and we haven't even begun to fine tune it. i expect it will require human in the loop checks for the foreseeable future, but even with a human approval of each item, its going to save the clerical staff hundreds of hours per year.…

> its going to save the clerical staff hundreds of hours per year How many hundreds of hours is your team spending to get there? What is the ROI on this vs investing that money elsewhere?

Can't speak to the financial benefit over other investment. Total dev/testing time looks to be fairly small in comparison to time saved in even one year, although with different salaries etc I cannot be too certain on the money ratio. Ultimately not my direct concern, but those making decisions are very happy with results so far and looking for additional processes to apply this type of system to.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#362

Post by hiddencost »

eldenring wrote: post #29

This is how America ends up being ahead of the rest of world with every new technology breakthrough. They spend a lot of money, lose a lot of money, take risks, and then end up being too far for others to catch up. Trying to claim victory against AI/US Companies this early is a dangerous move.

Like solar power? Or electric cars? Or drones?

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#363

Post by Imustaskforhelp »

trenchpilgrim wrote: post #285
roywiggins wrote: post #278

Earlier quoted context omitted.

In the context of call centers in particular I actually can believe that a moderately inaccurate AI model could be better on average than harried humans writing a summary after the call. Could a human do better carefully working off a recording, absolutely, but that's not what needs to be compared against. It just has to be as good as a call center worker with 3-5 minutes working off their own memory of the call, not…

Especially humans whose jobs are performance-graded on how quickly they can start talking to the next customer.

Yeah Maybe that's fair in the current world we live in.

But the solution isn't to use AI instead of not trusting the agents / customer service rep because their performance is graded on how quickly they can start talking to next

The solution is to change the economics in the way that the workers are incentivized to write good summaries, maybe paying them more and not grading them in such a way will help.

I am imagining some company saying AI is good enough because they themselves are using the wrong grading technique and AI is best option in that. SO in that sense, AI just benchmarked maxxed in that if that makes sense. Man, I am not even kidding but I sometimes wonder how economies of scale can work so functionally different from common sense. Like it doesn't make sense at this point.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#364

Post by tux3 »

missedthecue wrote: post #343
tux3 wrote: post #96

Earlier quoted context omitted.

> it's a huge, measurable efficiency gain. > It's not going to replace anyone's job Mechanically, more efficiency means less people required for the same output. I understand there is no evidence that any other sentence can be written about jobs. Still, you should put more text in between those two sentences. Reading them so close together creates audible dissonance.

" Mechanically, more efficiency means less people required for the same output. " Why can't it mean more output with the same number of people? If I pay 100 people for 8 hours of labor a day, and after making some changes to our processes, the volume of work completed is up 10% per day, what is that if not an efficiency gain? What would you call it? It really depends on the amount of work. If the demand for your labo…

All else equal, the demand for support calls doesn't go up as your support becomes more efficient.

I get that we're trying to look for positive happy scenarios, but only considering the best possible world instead of the most likely world is bias. It's Optimistic in the sense of Voltaire.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#365

Post by missedthecue »

tux3 wrote: post #364
missedthecue wrote: post #343

Earlier quoted context omitted.

" Mechanically, more efficiency means less people required for the same output. " Why can't it mean more output with the same number of people? If I pay 100 people for 8 hours of labor a day, and after making some changes to our processes, the volume of work completed is up 10% per day, what is that if not an efficiency gain? What would you call it? It really depends on the amount of work. If the demand for your labo…

All else equal, the demand for support calls doesn't go up as your support becomes more efficient. I get that we're trying to look for positive happy scenarios, but only considering the best possible world instead of the most likely world is bias. It's Optimistic in the sense of Voltaire.

What i'm saying is that if the volume of support is high enough, and never even changed, it's completely possible to improve throughput without reducing demand for labor. The result is simply that you improve response times.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#366

Post by doorhammer »

adrr wrote: post #337
doorhammer wrote: post #205

Earlier quoted context omitted.

Sentiment analysis, nuanced categorization by issue, detecting new issues, tracking trends, etc, are the bread and butter of any data team at a f500 call center. I'm not going to say every project born out of that data makes good business sense (big enough companies have fluff everywhere), but ime anyway, projects grounded to that kind of data are typically some of the most straight-forward to concretely tie to a dol…

Those have been done for 10+ years. We were running sentiment analysis on email support to determine prioritization back in 2013. Also ran bayesian categorization to offer support reps quick responses/actions. Don't need expensive LLMs it.

Yeah, I was a QA data analyst supporting three multi-thousand agent call-centers for an F500 in 2012 and we were using phoneme matching for transcript categorization. It was definitely good enough for pretty nuanced analysis.

I'm not saying any given department should, by some objective measure, switch to LLMs and I actually default to a certain level of skepticism whenever my department talks about applications.

I'm just saying I can imagine plausible realities where an intelligent and competent person would choose to switch toward using LLMs in a call center context.

There are also a ton of plausible realities where someone is just riding the hype train gunning for the next promotion.

I think it's useful to talk about alternate strategies and how they might compare, but I'm personally just defaulting to assuming the OP made a reasonable decision and didn't want to write a novel to justify it (a trait I don't suffer from, apparently), vs assuming they just have no idea what they're doing.

Everyone is free to decide which assumed reality they want to respond to. I just have a different default.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#367

Post by [deleted] »

yifanl wrote: post #256
lacy_tinpot wrote: post #226

Earlier quoted context omitted.

It doesn't because we already know it doesn't because just like think about it and it clearly doesn't. So everyone trying to make the technology is a big dumb dumb for even trying. Like is the conclusion we shouldn't even try? This kind of thinking ridiculous.

Same reason why we aren't looking for innovative ways of using sugary soft drinks as a building material. Just because there's a non-zero chance we could come up with something isn't compelling enough by itself.

[deleted]

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#368

Post by nostrademons »

jordanb wrote: post #140
jawns wrote: post #56

Full disclosure: I'm currently in a leadership role on an AI engineering team, so it's in my best interest for AI to be perceived as driving value. Here's a relatively straightforward application of AI that is set to save my company millions of dollars annually. We operate large call centers, and agents were previously spending 3-5 minutes after each call writing manual summaries of the calls. We recently switched to…

We use Google meet and it has Gemini transcriptions of our meetings. They are hilariously inaccurate. They confuse who said what. They often invert the meaning "Joe said we should go with approach x" where Joe actually said we should not do X. It also lacks context causing it to "mishear" all of our internal jargon to "shit my iPhone said" levels.

I also use Gemini notes for all my meetings and find them quite helpful. The key insight is: they don’t have to be particularly accurate. Their primary purpose is to remind me (or the other participants) of what was discussed, what considerations were brought up, and what the eventual decision was. If it inverts the conclusion and forgets a “not”, we’re going to catch that, because we were all in the meeting too. It’s their to jog our memory of what was said, because it’s much easier to recognize correct information than recall it, it’s not the authoritative source of truth on the meeting.

This gets to a common misconception when it comes to GenAI uses: it functions best as “augmented intelligence” rather than “artificial intelligence”. Meaning that it’s at its best when there’s still a human in the loop and the AI supplements the parts the person are bad at rather than replacing the person entirely. We see this with coding, where AI is very good at writing scaffolding, large-scale refactoring, picking decent libraries, reading API docs and generating code that calls it appropriately, etc but still needs a human to give it very specific directions for anything subtle, and someone to review carefully for bugs and security holes.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#369

Post by Imustaskforhelp »

lotsofpulp wrote: post #107
dsr_ wrote: post #86

Earlier quoted context omitted.

Pro-tip: don't write the summary at all until you need it for evidence. Store the call audio at 24Kb/s Opus - that's 180KB per minute. After a year or whatever, delete the oldest audio. There, I've saved you more millions.

The summaries can help automate performance evaluation. If the employee disputes it, I imagine they pull up the audio to confirm.

the amount of false positives coming from wrongful AI summaries plus having to pull up the audio to confirm is so much more hassle than not using AI and evaluating on some different metric at the first place.

Seriously not kidding but the more I read these comments, the more I become horrified realizing wtf,The only reason I can think of integrating AI is because you wish to integrate AI. Nothing wrong with that, But unless proven otherwise through some benchmarks there is no way to justify AI.

So its like an experiment, they use AI and if it works/ saves time, great If not, then time to roll it.

But we do need to think about experiments logically and the way I am approaching it, its maybe good considering what customer service is now but man that's such a low standard that as customers we shouldn't really stand it. Call centres need to improve period. AI can't fix it. Its like man, we can do anything to save some $ for the shareholders. Only to then "invest" it proudly into AI so that they can say they have integrated AI and so they can have their valuations increased since VC's / stock market reacts differently to the sticker known as AI

man.. so saying that you use AI, should be a negative indicator instead of a positive one in the market and the whole bubble is gonna come crashing down when people realize it.

It physically hurts me now thinking about it once again. This loop of making humans bad for money, using that money for inferior product, using that inferior product only because you want AI sticker, because shareholders want valuation increase and the company is willing to do this all because they feel/ are rewarded for this by people who will buy anything AI related thinking its gold or maybe that more people will buy it from them at an even higher evaluation because AI sticker and so on..

Almost sounds like a pyramid.

Re: 95% of Companies See 'Zero Return' on $30B Generative AI Spend

#370

Post by tux3 »

missedthecue wrote: post #365
tux3 wrote: post #364

Earlier quoted context omitted.

All else equal, the demand for support calls doesn't go up as your support becomes more efficient. I get that we're trying to look for positive happy scenarios, but only considering the best possible world instead of the most likely world is bias. It's Optimistic in the sense of Voltaire.

What i'm saying is that if the volume of support is high enough, and never even changed, it's completely possible to improve throughput without reducing demand for labor. The result is simply that you improve response times.

But I think this comes back to the same question of understaffing/overwork. We have to ask what strategic thinking led to accept long response times in the past. And the answer is unequivocal.

Unless we're claiming there is an intractable qualified labor shortage in call centers, this is always the result of a much simpler explanation: it's much cheaper to understaff call centers

A company that wants to save money by adding more AI is a company that cares about cost cutting. Like most companies.

The strategy that caused the company to understaff have not changed. The result is that we go back to homeostasis, and less jobs are needed to reach the same deliberate target.

Post reply on HN