Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

this.weekinsecurity.com

Post reply on HN
Print view View original

261–270 of 287 posts

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#261

Post by ChuckMcM »

oenton wrote: post #205
ChuckMcM wrote: post #150

Earlier quoted context omitted.

I don't disagree, that pattern exists, but it is essentially true. Just not in the way the folks saying it is true understand it. If the "VC/ad/tech dark patterns, enshitification, privacy violations, dishonest marketing, Etc." wasn't allowed then their job might not exist. That can be true. What is missed is that if there is value in the thing, then it will exist. When I reflect back to someone making this argument…

"Its a very common rhetorical technique to argue that the status quo cannot be changed." Thank you for putting this so eloquently into words. This rigid thinking is also common in topics such as working conditions, collective bargaining, on-call time, parental leave, healthcare, and effectively (unintentionally or not) shuts down conversation. I've come to realize the objections from people who think this way all eff…

Thanks. For me, I was in debate club in High School and that included basic rhetoric. In college I took an argumentation class as a non-engineering elective. The most useful thing this class taught (for me) is how to 'see' the argument, and as a consequence see how it is constructed. Throughout my career it has been especially useful in "political" situations at work. Not everyone argues in good faith, and being able to spot those who are not is valuable.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#262

Post by ncr100 »

Cyan488 wrote: post #3

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.

"Marge, there's the truth..." (frowns and shakes head negatively) "...and there's THE TRUTH!" (smiles brightly and nods enthusiastically)

-Lionel Hutz, Simpsons, Season 9 - "Realty Bites"

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#264

Post by dbbk »

ludwik wrote: post #6
Cyan488 wrote: post #3

> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.

I like to dunk on Meta as much as the next guy, but I think this makes sense: deterministic verification like this is not, and should never be, the LLM’s job. The tools it has access to should enforce the permissions layer, ensuring that the LLM can never perform actions the user themselves should not be allowed to perform. In this case, the tool failed to do that.

Yes the LLM part is irrelevant here. It'd be just the same if it was a HTML form.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#265

Post by dbbk »

mvkel wrote: post #129
Lionga wrote: post #112

Earlier quoted context omitted.

One can only hope EU gives them a GDPR fine very close to the limit of 4% of global turnover. But when EU is actually need to protect customer I think they will fail.

Incidents like this show how unenforceable GDPR is, and how it's been a net negative for users since its inception. It's idealogical back-patting, toothless when it matters.

How is this unenforceable? If any EU citizens were hacked they're gonna come down like a ton of bricks on Meta Dublin.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#266

Post by nosioptar »

armoredkitten wrote: post #260
nosioptar wrote: post #259

Earlier quoted context omitted.

I tried asking about how my state treats violations of updoc. It replied with a long ass wall of text about how serious the Updoc statutes were in my state and how judges punish harshly. I pointed out that updoc is nonsense and asked why it didn't catch that. The answer was that it was my fault for giving it bad info.

what's updoc?

Not much, how're things going for you?

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#267

Post by TZubiri »

srdjanr wrote: post #254
TZubiri wrote: post #33

Earlier quoted context omitted.

I get the joke, but it's a relevant nuance that the new code, the chatbot, did not have 'the bug'. I still think that the mistake and head that should roll should be the one that published the chatbot. But it's important to acknowledge that there was a 'bug' in an underlying tool and not in the chatbot, and still PIP/fire those responsible for publishing the chatbot and exposed an otherwise internal tool to the publi…

Why should the chatbot team necessarily take the blame? For all we know, they could have got approval from the tool team to make it public, and passed additional security review for making it public. Also, why fire anyone after a single mistake?

I did mention PIP/fire, but to be fair, this looks like the worst security issue in the history of Meta, a company known for an almost impeccable cybersecurity clean sheet.

So yeah, firing somebody or a group of people is on the table. Especially when like 10% of the company was fired last week for unrelated reasons. If you are gonna do it, fire the people who slash the value of your company by billions of dollars.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#268

Post by cyc116 »

TZubiri wrote: post #267
srdjanr wrote: post #254

Earlier quoted context omitted.

Why should the chatbot team necessarily take the blame? For all we know, they could have got approval from the tool team to make it public, and passed additional security review for making it public. Also, why fire anyone after a single mistake?

I did mention PIP/fire, but to be fair, this looks like the worst security issue in the history of Meta, a company known for an almost impeccable cybersecurity clean sheet. So yeah, firing somebody or a group of people is on the table. Especially when like 10% of the company was fired last week for unrelated reasons. If you are gonna do it, fire the people who slash the value of your company by billions of dollars.

How not to do blameless postmortem lmao. Non of the the engineers involved in this incident had anything to do with the company-wide layoff. I'm deeply sorry if the layoff affected you. But blame firing/piping more engineers for an incident should NOT be on the table. The negative sentiment towards meta engineers on this post is just wild.

Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot

#270

Post by TZubiri »

cyc116 wrote: post #268
TZubiri wrote: post #267

Earlier quoted context omitted.

I did mention PIP/fire, but to be fair, this looks like the worst security issue in the history of Meta, a company known for an almost impeccable cybersecurity clean sheet. So yeah, firing somebody or a group of people is on the table. Especially when like 10% of the company was fired last week for unrelated reasons. If you are gonna do it, fire the people who slash the value of your company by billions of dollars.

How not to do blameless postmortem lmao. Non of the the engineers involved in this incident had anything to do with the company-wide layoff. I'm deeply sorry if the layoff affected you. But blame firing/piping more engineers for an incident should NOT be on the table. The negative sentiment towards meta engineers on this post is just wild.

>But blame firing/piping more engineers for an incident should NOT be on the table.

There has to be a level of fuck up where a resignation is appropriate, maybe this doesn't meet your bar, but surely you recognize that there exists a limit of incompetence that proves that one is not up to the demands for the job.

I used to be on your camp, blameless postmortems, the truth is more important than assigning blame and in all likelihood it's a systemic problem. But with time I realized two things, 1 there's actually incompetent people, 2 if you wrongly get blamed and you don't blame someone else, then it's your head that rolls, hate the game not the player, you have to assign blame to someone else if you are accused.

Post reply on HN