Air Canada is responsible for chatbot's mistake: B.C. tribunal • Hacker News Board

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#321

Post by ssnistfajen » Sat, Feb 17, 2024, 1:51 AM UTC

ndjshe3838 wrote: post #226

I hate the whole attitude the industry has towards AI models, it just seems so sloppy compared to software of the past It’s like “yeah we don’t really know what it’s going to do and it might screw up but whatever just launch it” Really surprised me that so many companies are willing to jump into LLMs without any guarantee that the output won’t be something completely insane

>it just seems so sloppy compared to software of the past

Corporate software has always been sloppy especially if said corporation isn't centered around said software (true for Air Canada)) and the technology is in an early adopter stage (true for LLM chatbots).

The decision makers aren't well-versed in these technologies themselves, because getting to where they are did not require knowing how to properly use those technologies.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#322

Post by Aeolun » Sat, Feb 17, 2024, 1:55 AM UTC

nicbou wrote: post #313

morkalork wrote: post #240

What are the chances that companies take the easy way out and slap a EULA on the chatbots like "by engaging in conversation with our automated agent you agree to "?

One thing I love about German law is that lopsided contracts are generally not valid.

I don’t think a “don’t trust anything our chatbot says” contract is very lopsided. You can just not use it. It’s not as if it’s a highly important facet of your life.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#323

Post by ashearer » Sat, Feb 17, 2024, 1:56 AM UTC

DavidSJ wrote: post #237

brentm wrote: post #216

I just can't believe it took a lawsuit to get an airline to fork over a few hundred bucks on something that was obviously their fault.

I think they weren't worried about those few hundred bucks so much as precedent resulting in some future liability of millions of dollars when their chatbot misinforms tens of thousands of customers about something.

On the other hand, if they had quietly offered a one-time policy exception before reaching the point of a court verdict, they would have avoided any kind of precedent.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#324

Post by makeitdouble » Sat, Feb 17, 2024, 1:58 AM UTC

thfuran wrote: post #292

londons_explore wrote: post #242

Earlier quoted context omitted.

I think eventually courts will decide that whatever a chatbot or employee says while working for the company is binding on the company, as long as there was no malicious intent on either the part of the employee or the customer.

If it hasn't happened yet for employees, why do you think it will happen for chatbots?

A clear "paper" trail.

What is described here do happen when employees send mails with explicit promises, but gets harder when only the company has proof of the exchange (recording of the call). Chatbots bridge that gap.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#325

Post by iainctduncan » Sat, Feb 17, 2024, 1:58 AM UTC

I am Canadian. Air Canada has the worst customer service and treatment by far. This is hardly surprising, they try to weasel out of any responsibility all the time.

This is the same company that just got slapped for making a disabled dude crawl out of the airplane when air cans special chair didn't show up. I shit you not.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#326

Post by itsoktocry » Sat, Feb 17, 2024, 1:58 AM UTC

wongarsu wrote: post #308

steveBK123 wrote: post #260

Earlier quoted context omitted.

Yes, I think people underestimate the amount of imagined LLM use cases that require accurate responses. To the point that hallucinations will cost money in fines & lawsuits. This is a new frontier in short sighted customer service staffing (non-staffing in this case). The people who are on the frontline communicating with customers can convert unhappy customers to repeat customers, or into ex-customers. There's a few…

It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuits. The bar LLMs have to clear to beat the average front line support operations isn't that high, as your own experience shows. And compared to a large force of badly paid humans with high turnover, LLMs are pretty consistent and easy to train to an adequate level. They won't beat great costumer…

>It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuit

A human will be more likely to say "I don't know" or pass you along, rather than outright lie.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#327

Post by mynameisvlad » Sat, Feb 17, 2024, 1:59 AM UTC

wongarsu wrote: post #308

steveBK123 wrote: post #260

Earlier quoted context omitted.

Yes, I think people underestimate the amount of imagined LLM use cases that require accurate responses. To the point that hallucinations will cost money in fines & lawsuits. This is a new frontier in short sighted customer service staffing (non-staffing in this case). The people who are on the frontline communicating with customers can convert unhappy customers to repeat customers, or into ex-customers. There's a few…

It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuits. The bar LLMs have to clear to beat the average front line support operations isn't that high, as your own experience shows. And compared to a large force of badly paid humans with high turnover, LLMs are pretty consistent and easy to train to an adequate level. They won't beat great costumer…

Sure, but there is such a thing as “the human element”. Humans aren’t perfect, and that is the expectation. That is not the same case with computers.

And especially for something where it’s just pulling data from an internal system. There is absolutely no reason to invent made up information and saying “well humans do it all the time” is just an excuse.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#328

Post by AdamJacobMuller » Sat, Feb 17, 2024, 2:00 AM UTC

wongarsu wrote: post #308

steveBK123 wrote: post #260

Earlier quoted context omitted.

Yes, I think people underestimate the amount of imagined LLM use cases that require accurate responses. To the point that hallucinations will cost money in fines & lawsuits. This is a new frontier in short sighted customer service staffing (non-staffing in this case). The people who are on the frontline communicating with customers can convert unhappy customers to repeat customers, or into ex-customers. There's a few…

It's not like human call center staff has never given anyone wrong information, or cost companies money in fines and lawsuits. The bar LLMs have to clear to beat the average front line support operations isn't that high, as your own experience shows. And compared to a large force of badly paid humans with high turnover, LLMs are pretty consistent and easy to train to an adequate level. They won't beat great costumer…

The bar isn't even that high.

They only need to increase the lawsuit/settlement amount by less than the amount the companies saves by automation.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#329

Post by sebastiennight » Sat, Feb 17, 2024, 2:00 AM UTC

TechSquidTV wrote: post #296

steveBK123 wrote: post #260

Earlier quoted context omitted.

Yes, I think people underestimate the amount of imagined LLM use cases that require accurate responses. To the point that hallucinations will cost money in fines & lawsuits. This is a new frontier in short sighted customer service staffing (non-staffing in this case). The people who are on the frontline communicating with customers can convert unhappy customers to repeat customers, or into ex-customers. There's a few…

With RAG it's entirely possible to essentially eliminate 100% of hallucinations, given you are ok with responding with "I don't know" once in a while. These situations are likely coming from poorly implemented chatbot, or they decided that "I dont know" was not acceptable, and really that should be a queue to send you to a real human.

This claim seems wildly inaccurate, as even with GPT-4 in a single conversation thread with previous human-written answers included, a repeat of a similar question just resulted - in my testing today - in a completely hallucinated answer.

I think your claim might be based on anecdotal testing. (I used to have that same feeling after my first implementation of RAG)... Once you get a few thousand users running RAG-based conversations, you quickly see that it's "good enough to be useful", but far from being as dreamy as promised.

Re: Air Canada is responsible for chatbot's mistake: B.C. tribunal

#330

Post by tialaramex » Sat, Feb 17, 2024, 2:00 AM UTC

ianferrel wrote: post #267

londons_explore wrote: post #242

Earlier quoted context omitted.

I think eventually courts will decide that whatever a chatbot or employee says while working for the company is binding on the company, as long as there was no malicious intent on either the part of the employee or the customer.

Probably not "whatever", but anything that passes a "reasonableness" standard. It's totally reasonable to expect that if an airline offers bereavement fares that you could file the paperwork a month later because, like, there's a lot going on if you need a bereavement fare and getting the paperwork together can take time. There are lots of things that an employee might say that would not be reasonable, even if they h…

Yeah, reasonableness is the usual requirement in law unless otherwise specified. The ruling we're talking about here uses reasonableness several times:

"Generally, the applicable standard of care requires a company to take reasonable care to ensure their representations are accurate and not misleading."

"I find Air Canada did not take reasonable care to ensure its chatbot was accurate."

"Mr. Moffatt says, and I accept, that they relied upon the chatbot to provide accurate information. I find that was reasonable in the circumstances."