Live data from Hacker News

Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

this.weekinsecurity.com

71–80 of 287 posts

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#71
post #49

"abusing" by using it's built in insecurity to do insecure things. It's like, people abusing an open door. "Guys, just because we left the door open to your bedroom doesn't mean we're responsible". God can only hope this is a business ending lawsuit.

It won’t be. also this is more like them leaving the keys in the door, then someone comes along, uses the keys, and steals all your stuff. truthfully, no equipment is actually defective in this scenario eh?

If we're nitpicking metaphors I think it's more accurate to say it's like they were taking requests to rekey the lock on your door.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#72
post #51
post #26

Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?

The nature of the invention is for people to relieve themselves of the burden of having to use their minds. And while there will be exceptions, (including, I'm sure you: the person reading this comment,) the vast majority of people are hungry to use AI in that spirit of being able to be lazy.

Lazy can be a good thing. Since time and attention are finite and not fungible, it allows you to do something else. There's a reason we're all too lazy to do long arithmetic with pen and paper, instead relieving the burden of using our minds by outsourcing to spreadsheets and calculators. Not only does it allow us to think at a higher level of abstraction, but it also means we can take our kids to the park more often.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#73
post #32
post #26

Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?

In their defense, they asked the LLM to make no mistakes

And it did no mistakes. System worked exactly as LLM intended.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#74
post #59

Earlier quoted context omitted.

In that case, the statement is so meaningless as to be useless. Why should we care how Meta splits up their microservices? The tool still failed. They just want to redefine the "tool" as something else, anything else, to avoid having to admit something negative about their precious AI. > The LLM correctly generated tokens according to user input, however due to a bug in a separate code path, the system did not proper…

I mean, I think many of us are curious and enjoy hearing more details about how and where bugs like this occur. What's wrong with that?

I'd love to read a proper technical post-mortem, but this obviously isn't it. It's a carefully-worded statement from a lawyer meant to minimize liability and reputational damage to the company.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#75

Earlier quoted context omitted.

I was reminded of the Murray Walker quote. “There's nothing wrong with the car except it's on fire”

My dad says, "But other than that, Mrs. Lincoln, how was the play?" (Usually said jocularly when everyone is at their most upset, e.g. a vacation ruined)

A friend said at one of those moments, "And other than that, how was the play Mrs Lincoln?" And the 3rd person replied, "I don't know, I've never seen the play 'Mrs Lincoln'"

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#76
post #52

I really hope this accelerates meta's decline. The world will adapt just fine without social media.

Realistically, how will this affect Meta at all? Some people are pissed, nobody else cares, business as usual.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#77
post #35

>AI-assisted account recovery system oh no...Meta what are you doing

Account recovery is by far the #1 kind of ticket any service will get. Either because people forget their credentials, lose their credentials, get hacked or get impersonated - and that's just the legitimate tickets, on top of that come illegitimate tickets from everyday script kiddies over ransom extortioners (i.e. the people that aim to steal "valuable" handles) to nation-state actors that, say, want to get access to DMs of people messaging oppositional accounts.

That in turn means three things... it costs a lot of money to have humans look at these tickets, the PR damage from both acting and not acting on such requests can be immense, and users/customers can be anything from the smartest and richest people on the world down to the kind of utter imbeciles whose brains get surpassed by bears [1] or who plainly are not able to write. To make it worse, often enough online services don't have any kind of tie back to some known government-issued ID (either directly or by a proxy such as a mobile phone SIM), there's corruption involved on all levels, and for particularly "juicy" targets the stakes, if they can be converted to a monetary amount at all, can reach into the millions of dollars.

Now, Instagram alone has 3 billion (!) users from across the world, so they are bound to not just having to spend a lot of money on user support (remember, we are talking about the entire world, they also need to deal with about 7.000 (!) actively spoken languages, and having attack targets that are as powerful as US Presidents or as rich as Elon Musk. Clearly, the risk management involved in the entire idea was horribly deficient, but let's not act like this is a trivial problem domain in the first place. And hence the push for AI, simply because it - if done correctly - can take a lot of work off of the first-level support desks for a fraction of the money.

[1] https://velvetshark.com/til/til-smartest-bears-dumbest-touri...

[2] https://www.sapiens.org/language/world-languages-counting-me...

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#78
post #51

Earlier quoted context omitted.

The nature of the invention is for people to relieve themselves of the burden of having to use their minds. And while there will be exceptions, (including, I'm sure you: the person reading this comment,) the vast majority of people are hungry to use AI in that spirit of being able to be lazy.

Lazy can be a good thing. Since time and attention are finite and not fungible, it allows you to do something else. There's a reason we're all too lazy to do long arithmetic with pen and paper, instead relieving the burden of using our minds by outsourcing to spreadsheets and calculators. Not only does it allow us to think at a higher level of abstraction, but it also means we can take our kids to the park more often…

https://thethreevirtues.com paraphrases something Larry Wall wrote in Programming Perl:

> If we’re going to talk about good software design, we have to talk about Laziness, Impatience, and Hubris, the basis of good software design.

sourced from https://bcantrill.dtrace.org/2026/04/12/the-peril-of-lazines..., where Bryan Cantrill makes the point that:

> The problem is that LLMs inherently lack the virtue of laziness. Work costs nothing to an LLM. LLMs do not feel a need to optimize for their own (or anyone’s) future time, and will happily dump more and more onto a layercake of garbage.

which I think is interesting, albeit somewhat tangential to the current discussion.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#79
post #66

Earlier quoted context omitted.

Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" ( https://www.youtube.com/watch?v=3m5qxZm_JqM ) I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.

Does it matter if the response is tone deaf or simply misguided? I am a bit nihilistic here, but in one week absolutely nobody will be talking about this. Are the affected individuals going to abandon instagram? Are people going to reduce their usage out of concern for the safety of their accounts? Nothing will happen, hence there is no need for actual humans writing a good, well intended response.

> Does it matter if the response is tone deaf or simply misguided?

I agree with you that in a week nobody will be talking any more, but I'm pretty sure it's a GDPR data breach, and they can have some trouble within EU.

Yeah, they probably don't give a fu.. about EU, but if the response doesn't matter at all why did they spend time on it?

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#80
post #29

Meanwhile an account I created for a new product was permanently disabled by an automated system with no path for me to appeal to a human. (If anyone at Meta/Instagram sees this I wrote a brief blog post with the details. Please help! https://addisonwebb.com/blog/2026-06-05-Can%20Someone%20at%2... )

This is extremely common, unfortunately, to a point where it's a known/expected outcome when you're first creating a brand or product page among those in the biz.

If this doesn't work, I'd encourage you to reach out to a brand/ad agency and pay them $100 to ask their meta contact to help you get unblocked. You pretty much have to know someone who knows someone at meta in order to create these.

Tip: Do not post about this on twitter or other platforms - you'll get a ton of automated spam.

Post reply on HN