Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?
[deleted]
Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
141–150 of 287 posts
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#142Earlier quoted context omitted.
No fan of Meta, but I think "staggering" is properly determined by the percent of users affected rather than the absolute number. It's staggering to an SMB with 100k customers; it's bad, but not "staggering" to an internet juggernaught with 3B MAU.
Twenty _thousand_ people had their personal data stolen, many of them relied on these accounts to run their business, many put at risk of hackers impersonating them. Meta in a fair world should be forced to financially compensate these people. They built a world where many people basically have to use their products for their jobs and then failed to look after the data because they wanted to replace customer support…
Over forty _thousand_ people die every year in the US from car accidents. Plenty of other preventable injustices happen in all areas of life. I wonder how many fathers are unjustly taken away from their children by a corrupt family court system, how many people die of treatable diseases denied treatment by insurance companies, how many kids lose interest in school because of bad teachers, how many customer service workers endure daily abuse because they need the job.
It's not that the breach isn't bad, or that Meta is a sympathetic company. It's bad and they're not. I just find it hard to feel outraged about this particular incident affected 1 out of every 10k users of a social media site when we live in a world with citizen's united, qualified immunity, and $300 insulin.
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#143Earlier quoted context omitted.
Software can be copied infinitely, so even $1 of liability is effectively infinite since an unlimited number of people can potentially use it and sue you when it blows up. Nobody's going to be distributing software on the internet for free if the cost of insurance alone precludes that.
This is not how liability works, anywhere. So I write a piece of code that "makes your screen do cool things" and it causes the power supply to fail on those screens. Someone reports that bug to me and I check it out and say "Oh, shit it does break power supplies." Then I immediately put a notice on and in the code that says "WARNING: This code will break the power supply of your montitor." And I put that warning in…
There's a pattern I noticed, especially on this site, where people claim various VC/ad/tech dark patterns, enshitification, privacy violations, dishonest marketing, etc MUST be allowed, otherwise open source or 'the internet' will face some sort of existential risk.
No bro - open source and the internet existed long before SV tech parasitism did and will exist long after.
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#144Earlier quoted context omitted.
Twenty _thousand_ people had their personal data stolen, many of them relied on these accounts to run their business, many put at risk of hackers impersonating them. Meta in a fair world should be forced to financially compensate these people. They built a world where many people basically have to use their products for their jobs and then failed to look after the data because they wanted to replace customer support…
Over forty _thousand_ people die every year in the US from car accidents. Plenty of other preventable injustices happen in all areas of life. I wonder how many fathers are unjustly taken away from their children by a corrupt family court system, how many people die of treatable diseases denied treatment by insurance companies, how many kids lose interest in school because of bad teachers, how many customer service wo…
The US car deaths stat is also completely insane and way higher than other countries. I can recognize that at scale, securing every account is a very difficult task, but with scale comes responsibility.
Meta plays fast and loose rushing in unsupervised vibeslop agents to save a penny. They should be significantly penalized for such a massive failure, particularly for how long this exploit was live and for how the victims were unable to get in contact with any human at Meta to restore their account.
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#145Earlier quoted context omitted.
The tool worked correctly and as intended, but due to a bug it did not work correctly nor as intended.
Sounds like they are saying the agent did not malfunction, and this vuln could have been triggered by a human support agent too.
Kind of interesting that LLMs are basically being sold as having “human-like” reasoning capabilities, but in this case when “obamawhitehouse” asked to have it’s password reset sent to bob12345667@gmail.com the LLM didn’t question it and just triggered the process that happened to have a bug.
Humans support agents certainly fall prey to social engineering all the time, but I can’t think of a case where it was done on this scale so easily.
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#146Earlier quoted context omitted.
Over forty _thousand_ people die every year in the US from car accidents. Plenty of other preventable injustices happen in all areas of life. I wonder how many fathers are unjustly taken away from their children by a corrupt family court system, how many people die of treatable diseases denied treatment by insurance companies, how many kids lose interest in school because of bad teachers, how many customer service wo…
The US car deaths stat is also completely insane and way higher than other countries. I can recognize that at scale, securing every account is a very difficult task, but with scale comes responsibility. Meta plays fast and loose rushing in unsupervised vibeslop agents to save a penny. They should be significantly penalized for such a massive failure, particularly for how long this exploit was live and for how the vic…
1.2M car-related deaths worldwide every year. WW1 worth every decade.
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#147Earlier quoted context omitted.
They should add a feature called "auto-really" that just automatically says "really?" after the chatbot answers a question to check if it's going to 180 upon this tiniest bit of scrutinity.
You joke but this is almost literally what Chain-of-Thought does, at least in the early days. They basically just added "Wait," to the model's output and fed it back to the model iirc
This can't be a trillion dollar industry...
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#148People were reporting their accounts were being taken over with proper 2fa. Everyone had wondered how they hackers could take over accounts with little information, people were saying "inside job."
This is exactly the stupid explanation I expected. Your privacy and security. Meta. Serious Business.
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#149Earlier quoted context omitted.
Realistically, how will this affect Meta at all? Some people are pissed, nobody else cares, business as usual.
Well, these hacks targeted large influencer accounts. It could have more severe impact than 20k randomly selected accounts.
Large influencer accounts without two factor authentication...
The only useful reaction to this is to point and laugh.
Re: Meta confirms 1000s of Instagram accounts were hacked by abusing its AI chatbot
#150Earlier quoted context omitted.
This is not how liability works, anywhere. So I write a piece of code that "makes your screen do cool things" and it causes the power supply to fail on those screens. Someone reports that bug to me and I check it out and say "Oh, shit it does break power supplies." Then I immediately put a notice on and in the code that says "WARNING: This code will break the power supply of your montitor." And I put that warning in…
There's a pattern I noticed, especially on this site, where people claim various VC/ad/tech dark patterns, enshitification, privacy violations, dishonest marketing, etc MUST be allowed, otherwise open source or 'the internet' will face some sort of existential risk. No bro - open source and the internet existed long before SV tech parasitism did and will exist long after.
I don't disagree, that pattern exists, but it is essentially true. Just not in the way the folks saying it is true understand it. If the "VC/ad/tech dark patterns, enshitification, privacy violations, dishonest marketing, Etc." wasn't allowed then their job might not exist. That can be true. What is missed is that if there is value in the thing, then it will exist.
When I reflect back to someone making this argument by saying, "So your argument is that you make your living as a pick pocket, but if pick pocketing is made to be illegal, you won't be able to make a living." Which of course would only be true if they only thing they could do was 'be a pick pocket'. Its a very common rhetorical technique to argue that the status quo cannot be changed. All the arguments that "you'll put all coal miners out of business if you require only green energy" And yet the people, the miners themselves, will likely be fine. The firms might not, but there are other firms that could exist.
This isn't a new problem, or one specific to this web site, although it does get disproportionately hit because so many technology companies saw what Google started in the 2000's and said, "Man there is soooo many ways to get money for this." rather than, "Is this a reasonable way to make money? Sure it is 'perfectly legal' but is it right? Is it moral?" The type of person who thinks that something is "Only illegal if you get caught" is neither moral nor particularly concerned about what is right. And we got a lot of that type.