101–110 of 151 posts
Earlier quoted context omitted.
It'd be terribly compute inefficient to not share prefix caches (KV cache) across customers.
What is the probability that two customers will have exactly the same tokens in cache? Wouldnt it require using the exact same CLAUDE.md, skills, MCPs and context? After that it is even worse since the nondeterminism of LLMs and humans
In order Fable 5 has rejected: "Recipe for red-braised pork, I have pork shoulder" "Write up a framework for MCP patterns I can give to claude code" "explain the biomechanics of motion in c. elegans" (I get this one, I mostly did it to test and it's related to my hobby project) Do we get an extra day of functional Fable 5 because it's down?
Using a throwaway account for obvious reasons, but I’m very involved in this space using LLMs from multiple providers. I’m aware of at least two instances in which the intermediate infrastructure “swapped” responses, once impacting Claude models and once impacting GPT models, from two different providers. One gave us a proper postmortem in which their API gateway was incorrectly handling HTTP 100 status codes, puttin…
Every emergent behavior from these actors - whose claim to positive moral values is barely plausible - should be reported, discussed, dissected and critiqued early and often.
Sounds like a hallucination unless proven otherwise, even the leading LLMs can do those from time to time, and they will always appear plausible like that. Also could be the session having a lot previous context, like 800K+, which (I think) makes hallucinations more likely. Relevant comment from the OP which makes a hallucination more likely: > There is one tool call result that includes a string that printed a pathn…
Exactly. If you've never had an LLM (all models) suddenly start spouting nonsense in a completely different language...you haven't been using LLMs that much. They will go absolutely insane some % of the time.
Earlier quoted context omitted.
Very true, this was likely an attack. Worth noting that mr kettle has done a defcon talk nearly every year on some variant of this attack, the most recent one titled "HTTP/1.1 must die" because he rightfully believes that switching to the binary headers of http/2 (specifically in reverse proxy connections to upstream servers) is the only way to systematically prevent these.
I’ll be back next month with a load of fresh vectors in “Can AI Do Novel Security Research? Meet the HTTP Terminator” https://portswigger.net/research/talks?talkId=36 Maybe my last presentation on the topic! Possibly.
What I know for sure:
1.Stuff that has nothing to do with the current session got mixed in.
What guessing:
1.There's a minecraft.py file in the tool folder, and that might have triggered some hallucination.
2.Maybe data from some other project on the user's local machine got mixed in somehow.
3.Or it could be from another user's conversation.
Honestly, if I think about how the system actually works, I don't think it's pulling from another user's data. But other people say they've had issues like that, so I can't completely rule it out.
I saw this thing on YouTube once. When a bunch of users share the same system prompt, or prefix, the computation results get shared through something called a KV Cache. At least, that's what I understood. Not sure if I got it right. But if there's some bug in the hashmap that's supposed to keep those caches separate, then maybe multi-tenant memory management just broke down and that's what caused this. I mean, I can guess, but who knows. And honestly, even if that's exactly what happened, they'd never admit it.
At the end of the day, LLMs are just word predictors, right? They build up some kind of semantic space inside. So maybe the user's question just happened to be near Minecraft in that space. That's kind of what I think.
Thanks for the detailed report. We’re confident this is a hallucination but of course take these reports seriously and the team is looking into it. We’ll report back if anything turns up.