Live data from Hacker News

Man scammed after AI told him fake Facebook customer support number was real

cbc.ca

51–60 of 152 posts

Re: Man scammed after AI told him fake Facebook customer support number was real

#51
When I worked on a customer facing chatbot at my previous employer, we specifically wrote in the prompt "our customer service is not reachable by phone", and we tested that the chatbot was able to use that information and respond appropriately.

But I guess you can't expect a tiny startup like Facebook to invest money into having 1 employee part-time tweaking the prompt of the chatbot to respond appropriately to commonly recurring user questions.

Re: Man scammed after AI told him fake Facebook customer support number was real

#52
post #5

This one is pretty bad. This guy found a fake Facebook customer support phone number in a Google search, then asked the Meta AI chat in Facebook Messenger if the number he found was a real Facebook help line... and Meta AI said that it was. There's a screenshot of the chat in the article.

The bad thing is that people still think LLMs can be trusted at all . Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators".

> The bad thing is that people still think LLMs can be trusted at all.

LLMs are as trustworthy as humans.

Humans have been being wrong for about as long as we have been lying.

Whether you get information from a human or an LLM, check it.

I worry about the people who insist on credible sources rather than checking information for themselves. I think 80% or more of them are trolling me, but there are some who genuinely do not apply the Scientific Method to check facts in their everyday life. I truly feel sorry for them.

Re: Man scammed after AI told him fake Facebook customer support number was real

#53
post #5

This one is pretty bad. This guy found a fake Facebook customer support phone number in a Google search, then asked the Meta AI chat in Facebook Messenger if the number he found was a real Facebook help line... and Meta AI said that it was. There's a screenshot of the chat in the article.

The bad thing is that people still think LLMs can be trusted at all . Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators".

Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators"

"Not helping" seems a wild understatement. "Deceiving people into taking the wrong frame" seems more accurate.

Re: Man scammed after AI told him fake Facebook customer support number was real

#54

Earlier quoted context omitted.

The bad thing is that people still think LLMs can be trusted at all . Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators".

The general public is getting lied to constantly. HN users have a bit more context to see through the bullshit but the marketing getting pushed in people is that these AI tools are super genius incredible world changing tools that make everyone 100x more productive.

[flagged]

Re: Man scammed after AI told him fake Facebook customer support number was real

#55

Earlier quoted context omitted.

The bad thing is that people still think LLMs can be trusted at all . Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators".

> The bad thing is that people still think LLMs can be trusted at all. LLMs are as trustworthy as humans. Humans have been being wrong for about as long as we have been lying. Whether you get information from a human or an LLM, check it. I worry about the people who insist on credible sources rather than checking information for themselves. I think 80% or more of them are trolling me, but there are some who genuinely…

It would be nice to have a confidence level for pieces of information, like humans have

Re: Man scammed after AI told him fake Facebook customer support number was real

#56
Again and again we see that LLMs are great for creative output and terrible for anything where correctness matters. You should only use it for the latter scenarios when generating answers is slow/hard/expensive, but verification of answers is quick/easy/cheap. Probabilistic and non-deterministic answers have their place, but these companies marketing them in products need to do a better job expressing the limitations.

Re: Man scammed after AI told him fake Facebook customer support number was real

#57

Earlier quoted context omitted.

The bad thing is that people still think LLMs can be trusted at all . Companies integrating them into their offerings are not helping the public adopt the correct mental framing of these tools as "plausible text generators".

> The bad thing is that people still think LLMs can be trusted at all. LLMs are as trustworthy as humans. Humans have been being wrong for about as long as we have been lying. Whether you get information from a human or an LLM, check it. I worry about the people who insist on credible sources rather than checking information for themselves. I think 80% or more of them are trolling me, but there are some who genuinely…

This is not true. Sure, humans can lie or get things wrong. But normal people will also admit when they don't know something. LLMs tend not to admit when they don't know something, and they use an authoritative voice that sounds like they know what they're talking about. To an untrained person, this can easily be misleading.

Re: Man scammed after AI told him fake Facebook customer support number was real

#59
post #5

This one is pretty bad. This guy found a fake Facebook customer support phone number in a Google search, then asked the Meta AI chat in Facebook Messenger if the number he found was a real Facebook help line... and Meta AI said that it was. There's a screenshot of the chat in the article.

I mean that’s kind of on meta, as a customer I shouldn’t really have to care about the internals of the company. If a disgruntled employee lies to customers, that shouldn’t be the customers problem either. To me, that’s all just a statement by the company.

Re: Man scammed after AI told him fake Facebook customer support number was real

#60
I'm terrified of this happening to my elderly parents. It's why, even though it can be time consuming, I always have them run "tech support" issues (no matter how small) through me or my bro in law so some foreign scammer doesn't drain their accounts.
Post reply on HN