Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

wcax.com

Post reply on HN
Print view View original

131–140 of 174 posts

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#131

Post by criddell »

uncertainrhymes wrote: post #127
criddell wrote: post #27

This is my favorite AI customer service story: https://arstechnica.com/tech-policy/2024/02/air-canada-must-... > Air Canada essentially argued that “the chatbot is a separate legal entity that is responsible for its own actions”

That event happened in 2022. It keeps being tagged as "AI" when it was nothing more than a dumb front end for a document search. Yes, it still screwed up. And yes, Air Canada tried to weasel out of it, but even though it was a chatbot it had no LLM behind it.

It may not have been an LLM (nobody said it was) but it was definitely more than just a document search.

AI systems were in use for decades before LLMs rolled around.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#132

Post by benmorris »

I'm testing it in my business. I'm in a unique position of running a business totally unrelated to software dev, handled customer service personally for 15+ years. I developed a AI backed live chat app first a little over a year ago. It has handled ~6,000 chats since then so I have a ton of valuable data on what does and doesn't work (spoiler alert, it works really well).

I'm taking that information and building a voice version. I'm taking a lot of time to make sure the phone version works naturally and that is can address most common things people call in about. Just like my chat version, GPT 4o was barely good enough to do what I needed. Since GPT 5 and on it has been more than adequate. I expect the same thing over time with voice, Open AI is going to release the GPT-live to the API which is a noticeable improvement.

Some people are just offended at the idea of talking to an AI chatbot voice or phone, no matter how good it is. I don't know how good this Kinney Drugs bot was, but it's very easy to make something that works but ends up being very frustrating for customers.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#133

Post by hvb2 »

SoftTalker wrote: post #103
reaperducer wrote: post #60

Earlier quoted context omitted.

EVEN if the AI is a little rough around the edges with relational calls, just solving for transactional calls with no hold is a massive win. You keep saying that reducing call time is the "win." It seems like that the only metric you're measuring. If so, your product is a failure, not a success. This is HEALTHCARE . You don't get to do "rough around the edges." You don't get to dismiss edge cases. I also work in heal…

As if the techs making $15/hr answering the phone don't make mistakes?

But at least he's responsible. And we accept that humans make mistakes.

However, when the mistake is made by the replacement that cost one of your community members their job for a few pennies gain...

The bar for these systems is higher than the bar for humans.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#134

Post by taurath »

Oras wrote: post #50
jonathan-adly wrote: post #25

So - I run a company that does AI agents for pharmacy. We are at Series B now and growing well - series C in 9-12 months given growth. We just acquired a competitor of that vendor so very much an insider. First, The technology works, and it scales, but the whole bottleneck is domain expertise and implementation. These are expensive, and hard to scale. We hire pharmacists as project managers, that's how important doma…

Good for you and wishing you more success. You probably going to be down voted as sentiment here in HN is going against AI due to the pressure on jobs, which is completely understandable from emotional point. People tend to compare AI to perfection for some reasons, which is not the case with humans. How many errors happened due to human phone operators? In the last few days there was an article where humans scored 6…

With pharmacies, people are emotional because their perscriptions getting filled can literally mean their livelihood, their health or their life. I hoard medications because my insurance will only cover 30 day fills, and often it takes 2-4 days to call enough people for insurance to cover a refill. Its a common problem for older folks to take half of their perscribed medication from either cost, or inability to get through to the pharmacy.

You effectively have AI gatekeeping healthcare, as companies roll these out and also take away any escalation path that a human call center, or local pharmacy staff might've been able to handle.

I've been also working with AI for years now, and it is absolutely horrifying to me that any company would feel comfortable rolling it out - you're basically saying a 5% error rate is acceptable to you when you're providing a gatekeeping service that is life or death.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#135

Post by taurath »

fernly wrote: post #125
Marsymars wrote: post #68

Earlier quoted context omitted.

> People tend to compare AI to perfection for some reasons, which is not the case with humans. So I mostly avoid commenting on or pontificating about the value or lack thereof of AI, but the problem I have with companies rolling out "AI for whatever" isn't that sometimes AI gets "whatever" wrong, it's that the companies simultaneous gut the processes around correcting these mistakes. I'm currently dealing with a mont…

With that experience, why have you stayed with that provider? It would seem to be no harder, and quite satisfying, to open new accounts with another provider for you and your wife, and then cancel these accounts.

Cancelling a long term credit car can profoundly effect ones credit score sadly.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#136

Post by toomuchtodo »

jonathan-adly wrote: post #53
dylan604 wrote: post #36

Earlier quoted context omitted.

> We have hundreds of pharmacies with us now and they mostly successful and happy. Our biggest friction is customers being burned by vendors like this and writing off AI altogether as "not ready". This is such a great graph. Your company feels successful because your customer is happy yet the rest of the world would be just happy if your company died in a fire. This seems to also be just fine for companies like yours…

Very much clueless commentary. Retail pharmacies in the US are dying dozens a day. They don't have margin to support people answering the phone like tech companies with 80% margin do. Welcome to the real world where it is survival for these pharmacies not greed or shareholder value. They literally lose money filling your branded prescription even with no labor baked in. We are their lifeline. I know because I am not…

PBMs, private equity, and health insurance companies are what are crushing pharmacy margins. AI voice assistants will not last as a solution imho.

https://nashp.org/state-tracker/state-pharmacy-benefit-manag...

https://www.thebignewsletter.com/p/boom-the-break-up-of-big-...

https://www.thebignewsletter.com/p/inside-the-mafia-of-pharm...

https://www.thebignewsletter.com/p/the-real-reason-walgreens...

> But the real reason Walgreens, and the pharmacy business in general, is dying, is because of a failure to enforce antitrust laws against unfair business methods and illegal mergers. Elson touched on it when he mentioned lower reimbursement rates, but I don’t think people appreciate the full scope of what happened to Walgreens, and to the full pharmacy business in general. This is not a case of bad management, it’s a case of desperate management.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#137

Post by themgt »

HPsquared wrote: post #114
PhilipRoman wrote: post #77

Earlier quoted context omitted.

This is an underappreciated point when people talk about AI/chat integrations. If they put a claude-level model behind it, I'm sure it could resolve 99% of the situations faster and better than a human could, or at least realize its limitations when it can't. But of course that is too expensive so they often use extremely weak models that are literally worse than a good incremental full text search box.

I wonder what the price level (per token) needs to be, to match a minimum wage employee.

100x higher than it is now?

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#139

Post by asdff »

PhilipRoman wrote: post #77
dj_axl wrote: post #41

FWIW I know of a pharmacy network that's implementing AI. Not only does the AI make egregious errors like speaking in Spanish when the conversation is in English. It also has a limited context window for ground rules of 5000 tokens or less. So correcting all the various errors goes over the instructions limit. As a result if the implementation goes through it will be of poor quality. All I can think is that people wi…

This is an underappreciated point when people talk about AI/chat integrations. If they put a claude-level model behind it, I'm sure it could resolve 99% of the situations faster and better than a human could, or at least realize its limitations when it can't. But of course that is too expensive so they often use extremely weak models that are literally worse than a good incremental full text search box.

Another aspect of the human version is that the human version is deterministic. Call center workers have a flow that is pretty much a series of if else statements with appropriate edge cases already carved out and places to break out and escalate to another provider. These flows have been iterated over for some time and refined to this point.

AI unfortunately is not deterministic. It would be nice if you could actually use it deterministically in this way, set a seed such that the logic flow are preserved across all sessions well into the future unless the logic flow instructions are changed. They just don't work like that though. Even if you have a good harness they can go off the rails pretty easily and need coaxing to get back to appropriate context. That is a nonstarter for the typical caller of a call center. Functionally speaking just an automated call where you touch type through options would be equivocal to the human operator and that paradigm has replaced a lot of call center work already, with the human operator serving to smooth things over if the person is old or can't comprehend the options for whatever reason and starts yelling incoherently into the automated line (my actual strategy for getting human operators on the line).

Everything becomes a nail though when you have a hammer though, even more so when you are selling hammers. So expect to see AI in plenty of places where it makes no sense to belong given the supremacy of existing tooling. There is also no advocate for a lot of existing tooling unlike AI that has sales teams active in pursuit of new customers integrating this into their product.

Re: Kinney Drugs pulls back AI phone assistant after hundreds of customer complaints

#140

Post by Dotnaught »

dcrazy wrote: post #90
swatcoder wrote: post #65

Earlier quoted context omitted.

First, most tech companies of comparable size to your customers have negative margins and survive on investment cycles, and the tech companies that are bigger haven't offered human support to anyone in decades. (And indeed, everyone already resents this practice and those tech companies. Generally, sentiment is abyssmal.) I know you're just trying to illustrate the real plight of your customers, but it's a bad illust…

Microsoft offers human chat support for consumers, once you try their self-help tools: https://support.microsoft.com/en-us/contactus?ContactUsExper... Apple offers online, telephone, and of course in-person support for hardware and software: https://support.apple.com/?cid=mc-ols-general-homepage-marco... Amazon offers online and telephone customer support on your “my orders” page. Netflix offers phone and chat suppor…

>Apple offers online, telephone, and of course in-person support for hardware and software

The last time I waited on hold for Apple developer support, I got to enjoy two whole hours of free hold music. So they offer human support, just not very enthusiastically.

Post reply on HN