Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

wcax.com

Post reply on HN
Print view View original

71–80 of 166 posts

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#71

Post by artisinal »

jonathan-adly wrote: post #25

So - I run a company that does AI agents for pharmacy. We are at Series B now and growing well - series C in 9-12 months given growth. We just acquired a competitor of that vendor so very much an insider. First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important doma…

The pharmacies are happy because you are giving them higher profits.

Meanwhile you are royally fucking over the end user with your AI shit.

“You are absolutely right! You should indeed not give your 5 year old child 1000 milligrams of ibuprofen per hour”

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#72

Post by AndrewKemendo »

superb_dev wrote: post #64
AndrewKemendo wrote: post #33

Earlier quoted context omitted.

Explain to me then why pharmacies have almost exclusively used IVR based phone systems for pharmacy refills and prescription management for the last decade? Like they don’t even say the full name of the prescription they just tell you the first three letters and they’ll say “for the patient born on…” Current pharmacy IT systems are currently almost 100% automated so how are you claiming it’s the first use?

Do you actually use these systems? They can’t at all handle my case where half the time the drug is on back order or there some insurance non sense in the way. I need to talk to a human every time

Correct they fail in any out of boundary condition which is like 50% of cases in my experience

Even worse is that the human is usually just as worthless because the problem is often upstream at the insurance provider or regulation

This is something I’m finding with people complaining about AI systems the AI system in this case is not any worse than the previous IVR system was it just expanded the use cases and now people say oh it doesn’t work for those expanded use cases

There’s no mention whatsoever of the current absolute devastatingly bad state of all software across the world

It’s like people are mad when there’s an application that already sucks, someone attempts to use genAI, that fails and then they blame AI

The whole thing is absurd

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#73

Post by recursive »

munk-a wrote: post #45
8n4vidtmkvmk wrote: post #31

Earlier quoted context omitted.

Just curious, why are drug names such a disaster? Do LLMs not understand pronunciation keys?

Drug names are frequently BS loaded with strange consonants without singular pronunciation guidelines (sadly English isn't German) so humans already have a lot of variation in how things are pronounced - models then need to comprehend those variations, regional dialects (like those fuggin' ahs from Boston). It ends up being quite a lot of variation. As some examples: Wegovy, Ixempra, Qvar, Keflex

Wow, those names are actually real. They sound like fly-by-night compliance "brand" names on amazon sellers. How did those get approved by a multi-billion dollar marketing team?

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#74

Post by jonathan-adly »

reaperducer wrote: post #60
jonathan-adly wrote: post #42

Earlier quoted context omitted.

NPS typically improves because no one stays on hold for 20+ mins anymore. People have no ideas have pharmacies work. Pharmacies don't control how much they pay for drug, or how much they get paid. The margins are terrible and you can't hire enough techs to answer the phone at peak call volume. So - most people experience are terrible as they wait on hold for a really long time. EVEN if the AI is a little rough around…

EVEN if the AI is a little rough around the edges with relational calls, just solving for transactional calls with no hold is a massive win. You keep saying that reducing call time is the "win." It seems like that the only metric you're measuring. If so, your product is a failure, not a success. This is HEALTHCARE . You don't get to do "rough around the edges." You don't get to dismiss edge cases. I also work in heal…

Do you know what NPS is? That's how the comment started. Patients like it. Because the alternative is to stay on hold for 20+ minutes, and then get it wrong. (Or have no pharmacies except Optum's and Express Scripts).

Good for you - I want in the ICU delivering care during COVID as a frontline clinician. You don't get to pull this line.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#75

Post by Polizeiposaune »

8n4vidtmkvmk wrote: post #31
exizt88 wrote: post #16

As someone who's been doing AI call centres for a while, I'm not surprised. It's really, really hard to build voice AI that works. There are no open source solutions that work out of the box. But the biggest problem is ASR. WER is still atrocious even with SOTA models. When you add drug names and regional accents, it's a recipe for disaster.

Just curious, why are drug names such a disaster? Do LLMs not understand pronunciation keys?

Trademarks.

And the patients generally don't have a pronunciation key. Doctors prescribing might not, either.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#76

Post by Oras »

Marsymars wrote: post #68
Oras wrote: post #50

Earlier quoted context omitted.

Good for you and wishing you more success. You probably going to be down voted as sentiment here in HN is going against AI due to the pressure on jobs, which is completely understandable from emotional point. People tend to compare AI to perfection for some reasons, which is not the case with humans. How many errors happened due to human phone operators? In the last few days there was an article where humans scored 6…

> People tend to compare AI to perfection for some reasons, which is not the case with humans. So I mostly avoid commenting on or pontificating about the value or lack thereof of AI, but the problem I have with companies rolling out "AI for whatever" isn't that sometimes AI gets "whatever" wrong, it's that the companies simultaneous gut the processes around correcting these mistakes. I'm currently dealing with a mont…

I actually made another comment about lack of leadership causing many issues including exhausting dev teams.

What you describe is an execution failure not AI, these are basic NLP systems that could be easily configured with some dataset to make them better, but i will make a wild guess that the bank has incompetent manager who was pushing for some numbers to put in a slide to say “I” did this.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#77

Post by PhilipRoman »

dj_axl wrote: post #41

FWIW I know of a pharmacy network that's implementing AI. Not only does the AI make egregious errors like speaking in Spanish when the conversation is in English. It also has a limited context window for ground rules of 5000 tokens or less. So correcting all the various errors goes over the instructions limit. As a result if the implementation goes through it will be of poor quality. All I can think is that people wi…

This is an underappreciated point when people talk about AI/chat integrations. If they put a claude-level model behind it, I'm sure it could resolve 99% of the situations faster and better than a human could, or at least realize its limitations when it can't. But of course that is too expensive so they often use extremely weak models that are literally worse than a good incremental full text search box.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#78

Post by zucked »

newswasboring wrote: post #18

The desired speed of AI adoption is what is hindering AI adoption. Big companies are trying to sell this as magic, and it's not. It requires proper use to get anything useful out of this system. But that takes away the magic of it and the investors can't have that as the eventual share price only works if it is magic. Frontier labs will smother their product with their timelines.

This is probably the most pragmatic take here and the overall responses support it. There's a handful of practitioners who are seeing some implementation success. They probably had significant domain expertise and are several iterations deep. Then there's a bunch of people who are experiencing the "AI iS mAgIc" approach where LLMs are being shoehorned into workflows with very little effort given to make the process actually… work (well)… to satisfy investor-driven mania. This growing-pains stage will pass eventually and we'll hit some equilibrium.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#79

Post by taurath »

jdw64 wrote: post #8

I think of this as the kiosk economy. It's like how businesses introduce kiosks even when having a human employee would be more convenient, just for the sake of greater profit. The biggest problem with AI customer service is that a human employee would've let a minor issue slide without escalating it. But a chatbot often inflames the situation, and by the time the customer reaches a human agent, they're already furio…

If an AI representing a company I must do business with is trying to “empathize” with me to calm me down about not being able to access the services I pay them for, you better believe it’s going to make me and other customers much more upset.

Then the poor wretched line workers have to handle people for whom the companies are screwing over - there is no effective line upwards to anyone making the decisions to use these things other than an impotent “feedback survey”. You can’t “ask for AI’s manager”. This accountability sink is not only horrible customer service, when it’s life and death like “can I get the perscription that allows me to live” wild stuff starts to happen.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#80

Post by chrisjj »

jonathan-adly wrote: post #42
anon7725 wrote: post #34

Earlier quoted context omitted.

> We have hundreds of pharmacies with us now and they mostly successful and happy How happy are their patients?

NPS typically improves because no one stays on hold for 20+ mins anymore. People have no ideas have pharmacies work. Pharmacies don't control how much they pay for drug, or how much they get paid. The margins are terrible and you can't hire enough techs to answer the phone at peak call volume. So - most people experience are terrible as they wait on hold for a really long time. EVEN if the AI is a little rough around…

> EVEN if the AI is a little rough around the edges with relational calls, just solving for transactional calls

So "AI" treats relatonal as excluding transactional??

Post reply on HN