Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The newest Instagram “exploit” is the goofiest I've seen

0xsid.com

Post reply on HN
Print view View original

131–140 of 528 posts

Re: The newest Instagram “exploit” is the goofiest I've seen

#131

Post by jedberg »

Kwantuum wrote: post #127
jedberg wrote: post #122

Security 101 when changing the email of an account for any reason: email the old account and let it know the change happened. The weird thing is I know the Instagram security team, and they are top notch. I have a feeling this was vibe coded by someone outside of security and security wasn't looped in.

The fact that this can happen at all without the security team's knowledge is telling.

Probably not as telling as you think it is.

The security team at any organization is always considered an enemy to product and innovation. It wouldn't be surprising if management made it impossible for them to put in place the monitoring necessary to know this was happening. Especially at somewhere whose motto is "move fast and break things".

Re: The newest Instagram “exploit” is the goofiest I've seen

#132

Post by parable »

Ozzie_osman wrote: post #77

The ironic thing is I know several legitimate humans who have lost access to their accounts years/months ago, and have been dealing with support hell trying to get access back. Maybe they should have hacked themselves.

I've said this before, too. Several people I know have used various tricks and exploits to fix problems that support teams supposedly couldn't fix.

Re: The newest Instagram “exploit” is the goofiest I've seen

#133

Post by soerxpso »

aidenn0 wrote: post #76
dpark wrote: post #43

Earlier quoted context omitted.

This exploit has essentially nothing to do with AI and everything to do with a terribly designed account recovery flow. This exact same flow could have been (and may have been; I don’t know how much the chatbot here actually does) statically coded.

My impression is that AI didn't replace static code in this place; it replaced a person, who (hopefully) would have been suspicious about sending an account recovery code for e.g. "obamawhitehouse" to e.g. "bscurtu.alfamm.ro@gmail.com"

You're giving a lot of credit to the human alternative, especially considering that the attacker only needs to find one lazy human.

Re: The newest Instagram “exploit” is the goofiest I've seen

#134

Post by tencentshill »

simonw wrote: post #90

> All the Telegram groups have quieted down as Meta seems to have patched it already, but it appears this particular method was active for weeks, if not months. Is that for real? I find it hard to believe that an exploit THIS simple and easy to abuse managed to stay live for weeks or months.

When your job is on the line, you use AI like your boss tells you to. Implement the spec and move on. No time to think about security, if you delay this feature it's your ass.

Re: The newest Instagram “exploit” is the goofiest I've seen

#135

Post by dpark »

rob wrote: post #111
dpark wrote: post #91

Earlier quoted context omitted.

Maybe? I don’t know what logic was actually in the LLM vs it just using a bad tool. Unless I missed it, the article had no actual context on that either. This looks like a terrible design rather than an AI problem to me, though.

What would need to happen for it to be considered an AI problem to you?

Evidence that it was actually AI based logic and not just a chatbot interface sitting on top of a shitty design.

Re: The newest Instagram “exploit” is the goofiest I've seen

#137

Post by parable »

mrandish wrote: post #68

I get that account recovery for sites with hundreds of millions of users is a huge burden they're struggling to manage but I'm shocked they didn't restrict such loose verification to the >90% of lower value accounts that aren't worth stealing and keep the stricter verif on high-value accounts. The next obvious thing would be to let accounts the algorithm judges to be low-value still opt-in to strict verif. The vast m…

This is how account recovery procedures used to work at a certain gaming company. They used to train support agents on what makes an account high-value and apply additional scrutiny to those recovery cases, while letting low-value accounts be recovered with less information. It worked, for the most part, but because the valuation of a given account was based on the agent, some agents used to value accounts differently. You could get away with stealing a high-value account if you got the right agent in a support ticket. The tradeoff in this case was time spent - you'd have to create a lot of email addresses and plausible but vague tickets, though some attackers automated that process. Eventually, they just applied the same scrutiny level against every account and called it a day.

Re: The newest Instagram “exploit” is the goofiest I've seen

#138

Post by SCdF »

Jesus fucking Christ. On a bicycle.

LLMs should be treated as untrusted. At all times.

The mind boggles at the attitudes that seem to have have led to LLMs being an excuse to throw any of the "science" in computer science we've managed to get into production out the window and go elbow deep into treating computers like mystical alchemy.

The next decade is going to be a bumpy ride.

Re: The newest Instagram “exploit” is the goofiest I've seen

#139

Post by drtz »

hbn wrote: post #21

It's insane the AI has been provided the tooling to send emails to arbitrary addresses like that. Like, getting it to send a 2FA code at a user's request is one thing. But it should only be able to "hit a button" to send a 2FA email to the address attached to the account, all run with hand-written code. It shouldn't have access to the 2FA code itself, or the message subject, or body, or the recipient address, etc. Wh…

Yeah it's bad, but AI isn't required for this type of thing to work.

My anecdotal experience is my Facebook account was compromised several years ago after TOTP 2FA was disabled. Didn't exactly give me a warm fuzzy about Facebook security policies at the time, and this new attack just reaffirms that.

Re: The newest Instagram “exploit” is the goofiest I've seen

#140

Post by addaon »

afdbcreid wrote: post #83
dpark wrote: post #43

Earlier quoted context omitted.

This exploit has essentially nothing to do with AI and everything to do with a terribly designed account recovery flow. This exact same flow could have been (and may have been; I don’t know how much the chatbot here actually does) statically coded.

This is not true. Well, it kinda is, but nobody will be stupid enough to hand-code an account recovery where you get to type any email address. The reason it worked there is that the designers of the system didn't anticipate that the AI will agree to accept any email (maybe they even put guardrails against it in the system prompt, we don't know). It's more like social engineering than bad-security-code, except that l…

> The reason it worked there is that the designers of the system didn't anticipate that the AI will agree to accept any email (maybe they even put guardrails against it in the system prompt, we don't know).

These are contradictory cases. If you put guardrails into the system prompt, you've anticipated that the AI will take the action you're guardrailing against. And since AI prompt compliance is at best stochastic (and realistically just crap, over large sample sizes), every guardrail is an explicit recognition of a failure -- the guardrail will be ignored, and you can't pretend you didn't realize it was a problem, since you put it in.

Post reply on HN