Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Man scammed after AI told him fake Facebook customer support number was real

cbc.ca

Post reply on HN
Print view View original

71–80 of 152 posts

Re: Man scammed after AI told him fake Facebook customer support number was real

#71

Post by lxgr »

RecycledEle wrote: post #52
idle_zealot wrote: post #37

Earlier quoted context omitted.

The bad thing is that people still think LLMs can be trusted at all . Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators".

> The bad thing is that people still think LLMs can be trusted at all. LLMs are as trustworthy as humans. Humans have been being wrong for about as long as we have been lying. Whether you get information from a human or an LLM, check it. I worry about the people who insist on credible sources rather than checking information for themselves. I think 80% or more of them are trolling me, but there are some who genuinely…

Even if that were true (I don't think it is): The more important distinction between humans and LLMs is accountability.

If a customer support agent gives you incorrect information, you can often hold the company liable for it (assuming you can prove it; I suppose there's a reason for why companies prefer certain support channels over others).

If an AI "lies" to you, you're largely on your own right now.

Re: Man scammed after AI told him fake Facebook customer support number was real

#72

Post by lxgr »

Animats wrote: post #64
jessriedel wrote: post #30

Earlier quoted context omitted.

Yea, it’s certainly a reasonable argument if the wrong information comes from the company itself.

That's an excellent point. That court decided that an AI agent was an agent in the legal sense. "Agent" is a legal concept - someone acting for someone else.[1] It's what allows employees to act for a company. Otherwise nobody could do anything without signoff from the top. There are limits to agency, but it's a rule of reason thing - you can assume a store clerk has the authority to sell you stuff, and someone whose…

The situation here is slightly different, though. Meta's AI in their various products is explicitly marketed as an LLM chatbot, not as a customer support channel.

Whether they've been diligent enough in making that distinction (and whether that's even possible) will very likely be determined in court at some point.

Re: Man scammed after AI told him fake Facebook customer support number was real

#73

Post by boscillator »

janalsncm wrote: post #18
happypumpkin wrote: post #16

Earlier quoted context omitted.

I'm Gen-Z and talking to a human representative of a company makes me much more confident that something will happen as a result of my efforts (though still not certain). I scheduled an apartment viewing recently, and the only method they provided to do so was chatting with an AI (seriously)... I then tried and failed to find a way to contact a human for confirmation multiple times. Lo and behold nobody at the leasin…

When I was searching for apartments every complex had the same AI program for scheduling. It was horrible. I got to talk to one of the leasing managers at one of the viewings and I told him it made them seem cheaper, not more tech-savvy. He told me they had spent millions of dollars on it.

There seems to be a semi-infinite market for garbage software sold to landlords. At my current place I need an account to unlock my door, a different account to open the garage door (because the garage is managed by a third party), an account to reserve the elevator for move in day (which tried to up sell me moving services), an account to get sent my water bill which charges me $15 a month for the privilege (I don't pay me bill though this service, just have it emailed to me) , an account to pay rent and and an account to submit maintenance requests. Part of the trick seems to be to offload the costs onto the tenets who have no choice, but I'm sure our landlord is paying a good chunk for some of these.

If you have minimal to zero scruples, this seems to be an easy market to make a start up in. Landlords will buy anything!

Re: Man scammed after AI told him fake Facebook customer support number was real

#74

Post by dghlsakjg »

CoastalCoder wrote: post #20
simonw wrote: post #5

This one is pretty bad. This guy found a fake Facebook customer support phone number in a Google search, then asked the Meta AI chat in Facebook Messenger if the number he found was a real Facebook help line... and Meta AI said that it was. There's a screenshot of the chat in the article.

This reminds me of that recent issue with a Canadian airline, where (IIRC) a court ruled that their chatbot made a wrong, but binding, commitment to a customer. I'm curious if a Canadian court would hold Meta liable for the man's losses in this case as well.

That was a very interesting case. The chatbot in question was not LLM based (the incident was pre-chatGPT in any case) and was simply parroting an out of date or incorrect policy that it had been explicitly programmed to do. It seemed to gain a lot more traction in the press because of LLMs. "Air Canada forced to honor terms and conditions on their website" is a whole lot less interesting.

This FB thing is a case of an LLM simply hallucinating without direct human intervention.

Very different cases from a computer science perspective. My hope is that legally, they don't get viewed differently.

If you outsource functions of your business to a third party contractor you are still responsible for what they do and say. I don't think we should allow companies to weasel out of their obligations because they were dumb enough to let a sentence generator loose in a way that it could make commitments.

Re: Man scammed after AI told him fake Facebook customer support number was real

#75

Post by lxgr »

throwaway48476 wrote: post #42
JohnMakin wrote: post #34

Earlier quoted context omitted.

Bit tangential, but what the heck is it with scammers saying "dear" so much? Pretty much every pig butchering or social engineering attempt has had them repeatedly addressing me as "dear."

It fell out of fashion in western English speaking countries decades ago but not the 3rd world English speaking countries the scammers come from.

Many companies outsource their customer support staff as well.

That, and the fact that LLMs are now available to pretty much anyone for effectively nothing, would make me very cautious in basing my judgement of something being a scam or not exclusively on a caller's accent, spelling, mannerisms etc.

Lately, it's actually been quite the opposite in my experience, and I don't find that too surprising either: A lucrative scam business can afford to pay much more than the average US company that sees customer support as a cost center to be optimized at any cost. So why wouldn't their staff's English be better?

Social engineering scams are about to become a lot more exciting (in a bad way), not least thanks to LLMs (with and without voice capability), and I think people are absolutely not ready for it, not even us professionals working in tech.

Re: Man scammed after AI told him fake Facebook customer support number was real

#76

Post by mrweasel »

not_your_vase wrote: post #7

As a millenial, I'm more amazed that someone willingly uses a phone for non-mandatory and not-burningly-urgent phonecalls... why on earth would anyone do that is way beyond me.

Not sure why throwing in the randomly assigned label of millennial, but fine, I also fall in the category and I've taken to just calling people and companies.

First of all, understand that many especially smaller companies have people who has the job of answering phone calls. Rather than doing a multi day back and forth via email or chat where you're one out of five that "agent" is currently servicing, calling is really really efficient. Clarification and confirmations are instant, alternatives can be quickly discuses. I call because it's efficient.

Also, have you ever noticed that most people SUCK at email? Try sending an email to company with two or more questions. What will happen is that you'll get an answer for the first question and then they forget about the rest. The larger the company the more likely this is to happen, because they can deal with three issues in one support ticket, at least that's my theory. So now you need three email.

I used to hate calling people, but I found that I hated uncertainty more and I hate getting wrong half answers to my questions. Calling people fixes all of this. Always call, but get confirmation in writing.

Re: Man scammed after AI told him fake Facebook customer support number was real

#77

Post by Majromax »

lxgr wrote: post #71
RecycledEle wrote: post #52

Earlier quoted context omitted.

> The bad thing is that people still think LLMs can be trusted at all. LLMs are as trustworthy as humans. Humans have been being wrong for about as long as we have been lying. Whether you get information from a human or an LLM, check it. I worry about the people who insist on credible sources rather than checking information for themselves. I think 80% or more of them are trolling me, but there are some who genuinely…

Even if that were true (I don't think it is): The more important distinction between humans and LLMs is accountability. If a customer support agent gives you incorrect information, you can often hold the company liable for it (assuming you can prove it; I suppose there's a reason for why companies prefer certain support channels over others). If an AI "lies" to you, you're largely on your own right now.

Not necessarily. In Canada, a case in February (https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-...) held that Air Canada could be held responsible for incorrect information about a refund given to a customer by its chatbot.

Notwithstanding differences in jurisdiction, applying that idea to this case would rely on finding that Meta owed Gaudreau a duty of care that extended to the Meta AI chatbot.

It would be more difficult to make this claim if Gaudreau had asked the question of Google, since Google itself is not usually responsible for false information uncovered by its searches.

Re: Man scammed after AI told him fake Facebook customer support number was real

#78

Post by lxgr »

Majromax wrote: post #77
lxgr wrote: post #71

Earlier quoted context omitted.

Even if that were true (I don't think it is): The more important distinction between humans and LLMs is accountability. If a customer support agent gives you incorrect information, you can often hold the company liable for it (assuming you can prove it; I suppose there's a reason for why companies prefer certain support channels over others). If an AI "lies" to you, you're largely on your own right now.

Not necessarily. In Canada, a case in February ( https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-... ) held that Air Canada could be held responsible for incorrect information about a refund given to a customer by its chatbot. Notwithstanding differences in jurisdiction, applying that idea to this case would rely on finding that Meta owed Gaudreau a duty of care that extended to the Meta AI chatbot. It wo…

That's going to be indeed an interesting question (also discussed in this sibling thread: https://news.ycombinator.com/item?id=40536860).

My gut feeling is that it should be possible for companies to distinguish an AI product (i.e. as something provided to customers like a search engine, as you say) from an AI "working for them", but I can see a lot more disclaimers showing up in Meta's various AI chat channels soon.

Re: Man scammed after AI told him fake Facebook customer support number was real

#79

Post by jug »

Yes, AI in its current form is going to be a problem. I'm sure we haven't heard the worst yet. An AI may eventually kill a user.

I believe the heart of the problem is that corporations are riding a hype wave as long as they can, and an AI chat looks like super convincing, next level stuff thanks to the simple interface that hides the fact that you cannot communicate with this one as you would with a human being. You use natural language and it responds with natural language, which makes it not only convenient, but also dangerous.

There's money to gain on all this. While at the same time, hallucinations are an unsolved problem as well as making AI humble enough to realize and tell users that they just don't know. The combination of hallucinating, raising convincing arguments, being confidently incorrect, and not knowing the boundaries of your knowledge base, is a terrible one to let loose as officially sanctioned products.

Re: Man scammed after AI told him fake Facebook customer support number was real

#80

Post by lxgr »

josephcsible wrote: post #63
RecycledEle wrote: post #52

Earlier quoted context omitted.

> The bad thing is that people still think LLMs can be trusted at all. LLMs are as trustworthy as humans. Humans have been being wrong for about as long as we have been lying. Whether you get information from a human or an LLM, check it. I worry about the people who insist on credible sources rather than checking information for themselves. I think 80% or more of them are trolling me, but there are some who genuinely…

It's not about intentional deception. LLMs are very confidently incorrect way more often than humans are.

Eh, I've had the questionable pleasure of talking to first level support call centers a couple of times recently, and I wouldn't be so sure about that.

The number of times I've been told that yes, resetting my iPhone's network settings and reinstalling an app will resolve my billing issue or similar...

Post reply on HN