51–60 of 287 posts
Why was 'can a user request a different email' not literally the first test that comes to mind when making something like this? Do they not test anything because the scale is too big?
> "The tool itself worked properly and functioned as intended; however due to a bug in a separate code path, the system did not properly verify that the email address provided by the individual requesting a password reset matched the email address associated with that user’s Instagram account," said Meta in its breach notice. I'm not sure "worked properly" and "as intended" accurately describe this situation.
Earlier quoted context omitted.
I like to dunk on Meta as much as the next guy, but I think this makes sense: deterministic verification like this is not, and should never be, the LLM’s job. The tools it has access to should enforce the permissions layer, ensuring that the LLM can never perform actions the user themselves should not be allowed to perform. In this case, the tool failed to do that.
>deterministic verification like this is not, and should never be, the LLM’s job. But when humans handled it, this was not as much as a problem. That is, the humans did the job, because they recognized the need to do that job. Sure sometimes accounts could get recovered if a human was tricked, but evidently it was easier to trick the LLM in masse than humans.
In fact it's arguably a feature. The ability of support staff to short-circuit nitpicky rules when there's an obvious external validation happening (e.g. you're on the phone with a user who's presenting ID in real time and correlating it with previous use of the account, etc...) makes for better data quality and happier customers.
Obviously, yes, you can then human-engineer an authentication breach. But that was very difficult, because people are "common-sense careful" in a way we haven't been able to tease out of AI yet.
Corrected headline: "Meta confirms 1000s of Instagram accounts were hacked due to their insecure AI chatbot".
Earlier quoted context omitted.
In italian we say "l'operazione è riuscita perfettamente, ma il paziente è morto" -> "the surgery was a complete success, but the patient died"
Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" ( https://www.youtube.com/watch?v=3m5qxZm_JqM ) I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.
Earlier quoted context omitted.
Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" ( https://www.youtube.com/watch?v=3m5qxZm_JqM ) I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.
I was reminded of the Murray Walker quote. “There's nothing wrong with the car except it's on fire”
(Usually said jocularly when everyone is at their most upset, e.g. a vacation ruined)
Earlier quoted context omitted.
To be fair, that quote in the original article could have more context. By "The tool" they meant "AI-assisted support tool"[1]; perhaps they meant that the issue was not an AI hallucination inherent of the tool, but a fixable bug. [1]: https://www.documentcloud.org/documents/28202858-meta-ai-ag-...
In that case, the statement is so meaningless as to be useless. Why should we care how Meta splits up their microservices? The tool still failed. They just want to redefine the "tool" as something else, anything else, to avoid having to admit something negative about their precious AI. > The LLM correctly generated tokens according to user input, however due to a bug in a separate code path, the system did not proper…
Earlier quoted context omitted.
In italian we say "l'operazione è riuscita perfettamente, ma il paziente è morto" -> "the surgery was a complete success, but the patient died"
Both this and what Meta said reminds me of "Clarke and Dawe - The Front Fell Off" ( https://www.youtube.com/watch?v=3m5qxZm_JqM ) I also can't believe the people who were involved with writing this response from Meta, didn't realize how obviously bad it sounds. It's like there is no humans working and writing there anymore.
Don't know if AI is to blame, but I've used to see these kinds of nonsense post-mortems even in the pre-llm era, and it's always due to some internal fighting ongoing between various departments.