Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

wcax.com

Post reply on HN
Print view View original

121–130 of 180 posts

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#121

Post by MSFT_Edging »

jonathan-adly wrote: post #25

So - I run a company that does AI agents for pharmacy. We are at Series B now and growing well - series C in 9-12 months given growth. We just acquired a competitor of that vendor so very much an insider. First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important doma…

> First, The technology works

God I would love it if this comment is read out at a hearing about defrauding investors or massively leaking PII

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#122

Post by fusslo »

drewg123 wrote: post #107
jonathan-adly wrote: post #42

Earlier quoted context omitted.

NPS typically improves because no one stays on hold for 20+ mins anymore. People have no ideas have pharmacies work. Pharmacies don't control how much they pay for drug, or how much they get paid. The margins are terrible and you can't hire enough techs to answer the phone at peak call volume. So - most people experience are terrible as they wait on hold for a really long time. EVEN if the AI is a little rough around…

I'll gladly take a 20min hold after having to resort to screaming "give me a fucking human" or driving the the pharmacy JUST TO TALK TO a human in the exceptions, which seem to be about 75% of my interactions with CVS/Walgreens AI agents.

I recently had an AI chatbot lie to me. SiriusXM's ai bot INSISTED that the weather notifications were NOT a siriusxm feature. But, I had called ~4 months previously and the human confirmed it WAS a feature and she very kindly turned it off

Just as you said, I spent 20 minutes YELLING at the AI before it would transfer me to a human. Oh, and the entire time it was spewing ads for their programming. When I got to the human she, once again, confirmed it WAS a feature and it re-activated during 'free weekends'.

My father had cancer. After he was diagnosed he retired. His full time job became managing his health: medicare, state benefits, doctors appointments, medications, talking with billing people. He was on firstname basis with multiple providers. He invited some nurses and their families to the first 4th of july when he was cancer-free (they politely declined to keep the relationship professional).

I just imagine my father, or my 75yr old mother (whose health is beginning to decline) trying to navigate this new AI world. It must be an absolute nightmare. I can imagine how frustrated and disempowered they'd feel after a lifetime of reasoning with other humans.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#123

Post by lagrange77 »

drdeca wrote: post #98
lagrange77 wrote: post #44

Earlier quoted context omitted.

So you want him to make a piece of software that models a probability distribution rock solid.

Maybe “rock solid” could be understood as slightly hyperbolic (though it is metaphorical, so hard to say whether it is really hyperbole), but, it is possible to predict some things probabilistically with a high degree of accuracy. Depending on the task, a high enough degree of accuracy sounds doable, and if it offers a sufficiently convenient fallback to an actual human when it fails, that sounds like it could be qui…

> could be understood as slightly hyperbolic

Of course, you're right. I didn't intend to criticise him.

I just think that it's not a good idea to transfer this megatrend to healthcare.

Yes, those systems can be highly accurate, but as we all have experienced, this is not a stable or consistent property. A very good model can give you an ingenious answer one minute and an utterly dumb one the next. My guess is that it's simply a consequence of the extremely high complexity of the 'plant' (natural language, language interfacing with the real world), leading to some highly non linear behaviour.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#124

Post by Scoundreller »

cameldrv wrote: post #102
Oras wrote: post #76

Earlier quoted context omitted.

I actually made another comment about lack of leadership causing many issues including exhausting dev teams. What you describe is an execution failure not AI, these are basic NLP systems that could be easily configured with some dataset to make them better, but i will make a wild guess that the bank has incompetent manager who was pushing for some numbers to put in a slide to say “I” did this.

The parent commenter's experience pretty much sums up my experience with every IVR system, AI or not. The IVR cannot do anything I can't do on the website or the app. I wouldn't be calling if I could get the job done without calling them, so my experience is simply that the IVR is simply an obstacle to talking to a human. Very often the first human I talk to is simply an obstacle to talking to the second human, becau…

I suspect a lot of big orgs don’t care if they lose high-need/low-margin/negative-margin clients that need a couple chains of humans to fix their issue.

If they were otherwise high profit customers, they wouldn’t/shouldn’t have put them into IVR hell in the first place.

Anywho, I’m off to acquiring a DID that suggests I’m from a high-net-worth area. Until then, I’ll forge my caller id.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#125

Post by fernly »

Marsymars wrote: post #68
Oras wrote: post #50

Earlier quoted context omitted.

Good for you and wishing you more success. You probably going to be down voted as sentiment here in HN is going against AI due to the pressure on jobs, which is completely understandable from emotional point. People tend to compare AI to perfection for some reasons, which is not the case with humans. How many errors happened due to human phone operators? In the last few days there was an article where humans scored 6…

> People tend to compare AI to perfection for some reasons, which is not the case with humans. So I mostly avoid commenting on or pontificating about the value or lack thereof of AI, but the problem I have with companies rolling out "AI for whatever" isn't that sometimes AI gets "whatever" wrong, it's that the companies simultaneous gut the processes around correcting these mistakes. I'm currently dealing with a mont…

With that experience, why have you stayed with that provider? It would seem to be no harder, and quite satisfying, to open new accounts with another provider for you and your wife, and then cancel these accounts.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#126

Post by voicedYoda »

jonathan-adly wrote: post #25

So - I run a company that does AI agents for pharmacy. We are at Series B now and growing well - series C in 9-12 months given growth. We just acquired a competitor of that vendor so very much an insider. First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important doma…

I've been building agentic voice workflows with nearly 2 decades of healthcare experience. Would love to talk if you're available.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#127

Post by uncertainrhymes »

criddell wrote: post #27

This is my favorite AI customer service story: https://arstechnica.com/tech-policy/2024/02/air-canada-must-... > Air Canada essentially argued that “the chatbot is a separate legal entity that is responsible for its own actions”

That event happened in 2022. It keeps being tagged as "AI" when it was nothing more than a dumb front end for a document search.

Yes, it still screwed up. And yes, Air Canada tried to weasel out of it, but even though it was a chatbot it had no LLM behind it.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#128

Post by munk-a »

Marsymars wrote: post #88
munk-a wrote: post #37

Earlier quoted context omitted.

This, I suspect, is exactly like offshoring - and a lot of companies died during the offshoring fad because their software products become full of low quality merges and feature improvements that the company couldn't afford to fix or remove by the time they realized there was a problem. That remains my primary concern with AI code generation - well established codebases might become full of slop to the point of being…

There's precedence for new versions of software being abandoned for older code bases... usually the new version doesn't ship or gets forked relatively quickly though, e.g. the Windows Longhorn reset or the Gnome 3/MATE fork.

It can happen - and companies can recover from that. But that's wasting a whole lot of money and a lot of executives will sunk cost fallacy themselves into continuing when that may be the much saner decision.

I didn't mean to imply that all companies that went hard into AI will die - just that a lot of companies that follow the trend may not realize the true maintenance costs and of those that did misguidedly follow the trend I'm sure some will luck or manage their way out of disaster for various reasons.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#129

Post by munk-a »

dj_axl wrote: post #41

FWIW I know of a pharmacy network that's implementing AI. Not only does the AI make egregious errors like speaking in Spanish when the conversation is in English. It also has a limited context window for ground rules of 5000 tokens or less. So correcting all the various errors goes over the instructions limit. As a result if the implementation goes through it will be of poor quality. All I can think is that people wi…

This is probably (if the business here is large enough to support it) an excellent area for a tailored model instead of a general purpose LLM. A lot of the context around medication comprehension is going to be rather universal to pharmacies and other HCPs so ideally that information would be baked into the model itself with the actual user supplied context mostly being focused on business specific logic. The commodity LLMs are just an amalgam of a hammer welded to a screwdriver, a car, a compass and a toothbrush - more specialized models, with more specialized testing and offered by a company specialized in the field would possibly offer a decent customer experience here?

With something like healthcare though I'll remain eternally skeptical that anything other than a licensed expert will be palatable to the public.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#130

Post by Marsymars »

fernly wrote: post #125
Marsymars wrote: post #68

Earlier quoted context omitted.

> People tend to compare AI to perfection for some reasons, which is not the case with humans. So I mostly avoid commenting on or pontificating about the value or lack thereof of AI, but the problem I have with companies rolling out "AI for whatever" isn't that sometimes AI gets "whatever" wrong, it's that the companies simultaneous gut the processes around correcting these mistakes. I'm currently dealing with a mont…

With that experience, why have you stayed with that provider? It would seem to be no harder, and quite satisfying, to open new accounts with another provider for you and your wife, and then cancel these accounts.

Basically every provider seems to have a similar level of customer support.

For banking in particular, I already bank with a half-dozen banks because they have different products/services that are optimal for some specific use cases (i.e. I min-max my bank accounts, credit cards, and investment accounts.) — however, customer support seems to be universally bad.

If you're curious about why I'm using that particular bank, I'm forced to use them for some investment accounts because my employer matching benefits uses them. For the credit card, they're one of the few banks that offer proper co-applicants (rather than just supplemental users) which really works best for the way my wife and I have our finances organized. And flights in my city are dominated by a single airline where the only way to avoid paying for baggage is to get the co-branded credit card from this bank.

Post reply on HN