Live data from Hacker News

Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

this.weekinsecurity.com

251–260 of 287 posts

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#251
post #30
post #3

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.

In italian we say "l'operazione è riuscita perfettamente, ma il paziente è morto" -> "the surgery was a complete success, but the patient died"

We have the same saying in German: Operation erfolgreich, Patient tot.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#252
post #51
post #26

Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?

The nature of the invention is for people to relieve themselves of the burden of having to use their minds. And while there will be exceptions, (including, I'm sure you: the person reading this comment,) the vast majority of people are hungry to use AI in that spirit of being able to be lazy.

Unforseen side effect: I used AI to write so much code that now I struggle to have a good mental map of it, so I became lazy without having an intention to do so. Fun times!

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#254
post #33
post #16

Earlier quoted context omitted.

The tool worked correctly and as intended, but due to a bug it did not work correctly nor as intended.

I get the joke, but it's a relevant nuance that the new code, the chatbot, did not have 'the bug'. I still think that the mistake and head that should roll should be the one that published the chatbot. But it's important to acknowledge that there was a 'bug' in an underlying tool and not in the chatbot, and still PIP/fire those responsible for publishing the chatbot and exposed an otherwise internal tool to the publi…

Why should the chatbot team necessarily take the blame? For all we know, they could have got approval from the tool team to make it public, and passed additional security review for making it public.

Also, why fire anyone after a single mistake?

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#255

Earlier quoted context omitted.

Where do you think the LLMs learned it from...

"Who taught you how to do this stuff?" "You, alright! I learned it by watching you!"

Pretty much. The most depressing thing about the bland slop produced by LLMs is realizing that this is what they "think" of us.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#256

I'll never understand using AI/bot for customer support. IG is a well know platform. If I have an issue I feel pressed to connect with a support agent about it very likely is something a bot would struggle with, otherwise I'd just google. I understand there some grandmas who can do a google search, but the vast majority of folks reaching out for support are doing so because they have a real issue that can't be simply…

Unlike programmers, most people don’t read long help articles or take multi-step actions. Bots can answer questions directly and take actions (or guide a user through step by step).

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#257
post #194
post #30

Earlier quoted context omitted.

In italian we say "l'operazione è riuscita perfettamente, ma il paziente è morto" -> "the surgery was a complete success, but the patient died"

"Operation succesful, patient dead" is a common saying in India.

I found an English use from 1883 - https://archive.org/details/argonaut131883sanf/page/n391/mod... .

> The creosote in toothache drops administered to a New York boy cured the pain, but killed the boy. This recalls the entry in the register at Bellevue Hospital, which reads; "Operation successful. Patient died."

The Argonaut, San Francisco, December 22, 1883.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#258
post #3

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.

Unfortunately this statement will, in spite of what you identified correctly, likely do its job and divert attention from the fundamental issues we are facing with a technology that has already spread further than anyone can control. From enterprise too lay man. The whole world of computing was not built ever expecting software capabilities like this to ever exist.

I am not saying it's like a nuclear bomb. Rather like the first guns brought into fights the others were perfectly prepared for ti fight with swords and didn't even know yet, about this fascinating invention called a gun. Sounds interesting. Let me inspect it. Oh wow, that's interesting technology. What happens if i push that thing back? Will it re... oops...

Thank god that we have honourable people like altman, zuckerberg, musk. Imagine how bad all this would turn within the next few years, if major decisions were made by self-serving, delusional, greedy egomaniacs...

Of course currently let's first hope those wars and all the tension in societies all over the world, in war or peace, won't explode into something really, really bad. Looking at history, i fear we see how social tension on large scale over time... not saying it's not obvious to almost everyone. So well, let's just keep hoping. Maybe throwing blackbox AI tech into the mix, would surprise and change course of history. Actually, while i am thinking about it, i think i just changed my opinion into the opposite position, lol. Honestly, if it's 50/50 that this will lead to the worst possible outcome intensified, it's still better than just checking boxes following the "humans slowly stumbling into near-extinction experiences 101" handbook. Because just according to that, we're lucky if we're off by 10 years. There must be a big change in humanity and how the world is currently constructed, for all this leading to anything other than what we should expect from history. If we kept all nations busy with huge technological issues, that made all of their personal lifes so complicated, turn every elitists luxury into a burden, busy to defend what they own, while they can't realize, that normal life has changed so much, they now are the ones, frozen in life. They would have no time for conflict.

This sounds totally logical. In any other scenario, it would be pretty insane what we are all doing and entertaining (including me, top10 hypocrite).

I fear it's too late to turn ship, yet we still can jump ship.

---

Especially because now thinking about the thoughts that just went through my head, maybe (technological) disruptions are actually disrupting. But not a status quo of an economic model.

But a pretty clear loop of human nature and "humans in societies". And the more often we disrupt this loop, the more time we get before it's ready to start over again.

And now we have something that has the potential to change all fundamentals so much, that all the major conditions inside this loops iteration become meaningless. The environment changes so much, the state of the checkboxes gets emptied. Cache invalidated. Indices are gone.

Oh, i know how dumb this sounds. I am not even trying to claim anything. I didn't even think about it before, this is just a note of the words that i typed, almost on autopilot. No idea if i believe a part of this could be real. But even thought, just as a mere fictional story, it already entertained me.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#259
post #23
post #11

Earlier quoted context omitted.

That sounds a lot like the justifications Claude and ChatGPT give when confronted about something they did wrong, or when asked to provide a customer support response about software issues

I've lost track of the number of times Claude has basically said "it was like that when i got here" in the face of a clearly bogus choice and easily disproved explanation.

I tried asking about how my state treats violations of updoc. It replied with a long ass wall of text about how serious the Updoc statutes were in my state and how judges punish harshly.

I pointed out that updoc is nonsense and asked why it didn't catch that. The answer was that it was my fault for giving it bad info.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#260
post #23

Earlier quoted context omitted.

I've lost track of the number of times Claude has basically said "it was like that when i got here" in the face of a clearly bogus choice and easily disproved explanation.

I tried asking about how my state treats violations of updoc. It replied with a long ass wall of text about how serious the Updoc statutes were in my state and how judges punish harshly. I pointed out that updoc is nonsense and asked why it didn't catch that. The answer was that it was my fault for giving it bad info.

what's updoc?
Post reply on HN