Live data from Hacker News

The newest Instagram “exploit” is the goofiest I've seen

0xsid.com

71–80 of 528 posts

Re: The newest Instagram “exploit” is the goofiest I've seen

#71

I'm sitting here wondering why the Chief Master Sergeant of the U.S. Space Force has an Instagram account to begin with. I understand it's the office itself, but still don't see the reason to expand the attack surface of government offices. X makes sense, Instagram, I'm not so sure as much

I see no difference between X and Instagram in this regard whatsoever.

Think NASA, for example; it's also a government agency, and they are doing great job posting photos in Instagram, do you think anything is wrong with it?

Re: The newest Instagram “exploit” is the goofiest I've seen

#72

I'm sitting here wondering why the Chief Master Sergeant of the U.S. Space Force has an Instagram account to begin with. I understand it's the office itself, but still don't see the reason to expand the attack surface of government offices. X makes sense, Instagram, I'm not so sure as much

Outreach, I'd guess? You've got to do outreach where the people are. X and Instagram have pretty different audiences, but they're both large, so if you're on one you probably should be on both.

Re: The newest Instagram “exploit” is the goofiest I've seen

#73
post #36

Earlier quoted context omitted.

recovery is always the weakest link in any authentication system

This is not wrong but what’s really missing is cost: Meta did this so they can avoid paying people to do it. Lots of companies follow that decay spiral: your bank could shut phishers down cold by requiring wire transfers to be authorized in person but they don’t want to pay staff or risk you being upset by a transaction taking an extra hour so they don’t. Imagine an alternate universe where big tech companies worked…

Then you get trusted parties selling account access. Even if you remove them for a single false positive they will do it. A bit like a % packages "vanishing".

The least terrible seem digital id.

Re: The newest Instagram “exploit” is the goofiest I've seen

#74

How is this "embarrassing" instead of subject to legal liability? We really need similar rules to other engineering disciplines. If your building falls with people inside, you killed them.

Nobody dies if instagram collapses. Might even cause more people to live.

Re: The newest Instagram “exploit” is the goofiest I've seen

#75
post #21

It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc. Wh…

I do a lot of bug bounty research on Meta and Instagram, and some of the bugs I find look extremely simple like this but have some slightly complicated reason for why they occur. Maybe not this one, but I do have a guess as to what might have actually happened.

Based on what I've seen so far, Meta AI Support Assistant (they call it "MAISA") had tool calls that a) start an email verification to any specific email, phone number, or the contact points linked to an account and b) allow generating a password reset link for an account based on an email verification attempt. I don't think it had any access to the actual codes themselves, but rather think a handle or ID for an email verification attempt (along with the user provided verification code based on user input) was provided to the "generate reset password link" tool call, and the tool call failed to properly validate the actual email used in that attempt belonged to the account allowing the ATO.

The tool call for MAISA to generate a password reset link should have failed with an email verification attempt that corresponds to an email not linked to the account (and I believe I even tested this at one point on Facebook and encountered an error that successfully prevented it), but I suspect they tried making a change to this tool call for Instagram where slightly older, recently unlinked emails could be used to recover an account that got hijacked by an attacker, which added the need to allow emails not currently linked to the account to be used and set to the user's primary email.

I also suspect that the MAISA tool call change called a wrong API or something that unintentionally allowed any email verification attempt that was successful to be used, but the engineers did not add a sufficiently thorough e2e test case to test the tool call against unrelated email verification attempts being provided to the tool call. This is the part I think should be focused on the most. Tool calls for agents that have their output potentially influenced by an attacker should be treated like external APIs that anyone can reach, and they should be tested as such.

This is all obviously a guess, doesn't take into account the many signals they use to determine if an account recovery attempt is valid, and could be very inaccurate, but it's the closest to what I (someone who deals with Meta security a lot) think could have allowed this to happen.

Re: The newest Instagram “exploit” is the goofiest I've seen

#76
post #43
post #21

It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc. Wh…

This exploit has essentially nothing to do with AI and everything to do with a terribly designed account recovery flow. This exact same flow could have been (and may have been; I don’t know how much the chatbot here actually does) statically coded.

My impression is that AI didn't replace static code in this place; it replaced a person, who (hopefully) would have been suspicious about sending an account recovery code for e.g. "obamawhitehouse" to e.g. "bscurtu.alfamm.ro@gmail.com"

Re: The newest Instagram “exploit” is the goofiest I've seen

#78
post #9

Support requests have always been the weakest link in the security chain for big corps. I've had accounts of mine turned over with 2FA disabled by humans before. I guess we shouldn't be surprised that the LLMs are doing the same thing. The simple fact that 2FA can be removed by low level support staff drives me mad. It defeats the whole purpose of the process.

100% Urgency. Emotions. It's all there, and high-stakes environments with no proper protocol are most vulnerable. Source: used to work part-time in IT support at a hospital, by now 10+ years ago, so it was routinely requested to circumvent regulations and security protocols, even medical ones (cough Windows in ICU monitors and other medical "kiosk" PCs that should absolutely not run Windows)

I love those admin passwords which a tech will give you at some point because he doesn't want to do the work himself. If they even have passwords...

Unfortunately Siemens woke up.

Re: The newest Instagram “exploit” is the goofiest I've seen

#79
I'm among the first 6000 users of Instagram and my first name username was stolen a few years ago. Support for verified accounts acknowledged the issue, but couldn't do anything about it.

This turn was an AI exploit, in my case was an outsourcing support 'exploit', where someone paid for my username to be manually changed and given to another user. There will always be a way to get access to accounts if human accountable support doesn't exist, with criminal consequences for employees that violate it.

Re: The newest Instagram “exploit” is the goofiest I've seen

#80
This is an embarrassing failure for Instagram. But SIM cards have been hacked the same (by tricking support, claiming the phone was lost or stolen), except the agent was human.

The solution (which also solved SIM support agents being bribed or hacking known acquaintances) was to prevent the agents from resetting the SIM card without some steps the original owner would have to follow (and could follow even if they've lost their original phone), like a PIN they'd have to remember. I think the same solution should be applied to AI agents.

Post reply on HN