Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

this.weekinsecurity.com

Post reply on HN
Print view View original

181–190 of 287 posts

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#181

Post by xyst »

Cyan488 wrote: post #3

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.

It’s a public release prepped/reviewed by the in house legal counsel.

Don’t read too much into it. Facebook wants to face as little accountability and keep the future class action lawsuit to a minimum.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#182

Post by skyfaller »

prinny_ wrote: post #66
embedding-shape wrote: post #39

Earlier quoted context omitted.

Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" ( https://www.youtube.com/watch?v=3m5qxZm_JqM ) I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.

Does it matter if the response is tone deaf or simply misguided? I am a bit nihilistic here, but in one week absolutely nobody will be talking about this. Are the affected individuals going to abandon instagram? Are people going to reduce their usage out of concern for the safety of their accounts? Nothing will happen, hence there is no need for actual humans writing a good, well intended response.

Everyone has a limit to how much bullshit they'll put up with. This could be the last straw for some people to finally quit Instagram. I quit Instagram, Facebook and all other Meta properties in 2025, after complaining about various problems for years. Other people may quit temporarily and then return, but any time they spend away from Instagram may give them experience that will help them quit permanently later.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#183

Post by jjmarr »

Terr_ wrote: post #111
jjmarr wrote: post #99

Earlier quoted context omitted.

I've always wanted to expose myself to unlimited legal liability by distributing open source software.

That seems like a false-dichotomy between two extremes when there's all sorts of space in the middle... It's also assuming developer-to-developer tools would have the same rules and exposure as in service-to-consumer. If I sell a physical motor (let alone plans for one) I'll have some liability for things like it Not Exploding. If someone buys a dozen of those motors to assemble a tragically unsafe "rollercoaster" of…

The United States/Canada don't have a "loser pays" rule, so this exposes me to legal fees.

Right now, any lawsuit against me can be dismissed on summary judgement because even if my software causes harm, that's not a legal wrong to the extent I've disclaimed liability.

If you adopt any fact-specific standard for liability, that needs to be adjudicated in a trial. The legal fees alone would surpass the actual liability.

That creates huge leverage for the party with more resources. That kills hobbyist open-source development, since if your project takes off but a large enterprise finds it defective, they can threaten to sue you to enforce the "warranty" you were required to give.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#185

Post by gavmor »

ludwik wrote: post #6
Cyan488 wrote: post #3

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.

I like to dunk on Meta as much as the next guy, but I think this makes sense: deterministic verification like this is not, and should never be, the LLM’s job. The tools it has access to should enforce the permissions layer, ensuring that the LLM can never perform actions the user themselves should not be allowed to perform. In this case, the tool failed to do that.

The overall system that allowed this implementation is accountable. So why put such a fine point on it so as to exculpate the LLM?

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#186

Post by zahlman »

thih9 wrote: post #19
nkrisc wrote: post #16

Earlier quoted context omitted.

The tool worked correctly and as intended, but due to a bug it did not work correctly nor as intended.

To be fair, that quote in the original article could have more context. By "The tool" they meant "AI-assisted support tool"[1]; perhaps they meant that the issue was not an AI hallucination inherent of the tool, but a fixable bug. [1]: https://www.documentcloud.org/documents/28202858-meta-ai-ag-...

It seems to me like they're saying the agent made the tool call they expected, but the harness didn't reject it like they expected it to.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#187

Post by iamkrazy »

mikrotikker wrote: post #161
Gigachad wrote: post #133

Earlier quoted context omitted.

Twenty _thousand_ people had their personal data stolen, many of them relied on these accounts to run their business, many put at risk of hackers impersonating them. Meta in a fair world should be forced to financially compensate these people. They built a world where many people basically have to use their products for their jobs and then failed to look after the data because they wanted to replace customer support…

[flagged]

[dead]

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#188

Post by dylan604 »

maxall4 wrote: post #139
rothfuss wrote: post #42

Earlier quoted context omitted.

I was reminded of the Murray Walker quote. “There's nothing wrong with the car except it's on fire”

“The strait of Hormuz is open so long as Iran does not fire missiles at ships.”

"The numbers will go down as soon as you quit testing"

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#189

Post by justinclift »

prinny_ wrote: post #66
embedding-shape wrote: post #39

Earlier quoted context omitted.

Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" ( https://www.youtube.com/watch?v=3m5qxZm_JqM ) I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.

Does it matter if the response is tone deaf or simply misguided? I am a bit nihilistic here, but in one week absolutely nobody will be talking about this. Are the affected individuals going to abandon instagram? Are people going to reduce their usage out of concern for the safety of their accounts? Nothing will happen, hence there is no need for actual humans writing a good, well intended response.

> in one week absolutely nobody will be talking about this.

In news media, sure. But in IT teams around the world people will be referring to this (the exploit opening stupidity) for years as how NOT to do things. :)

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#190

Post by dylan604 »

tyleo wrote: post #152
Cyan488 wrote: post #3

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.

Error: Success!

You must work in QA
Post reply on HN