Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Potential session/cache leakage between workspace instances or consumer accounts

github.com

Post reply on HN
Print view View original

81–90 of 151 posts

Re: Potential session/cache leakage between workspace instances or consumer accounts

#81

Post by andy99 »

_def wrote: post #52

Reminds me of a session I had recently (on web!) where claude insisted that i prefixed all my messages with statements about code execution or something, which was not the case. I interrogated it about that and it confirmed that it came from somewhere else, but could not get rid of it and each response mentioned that its gonna ignore those instructions. Eerie.

Anthropic injects text into the conversation triggered by certain conversation topics. This happened to me in relation to some red-teaming related discussion that was adjacent to something “sensitive”, I think sex, and Claude got confused about why I had said some kind of warning and mentioned it it’s response. After a back and forth it was clear that some extra warning to answer but avoid anything inappropriate had been inserted into the conversation.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#83

Post by jstummbillig »

bri3d wrote: post #55
jstummbillig wrote: post #13

Is there anything particular about LLMs that would make separating customer data harder than in all SaaS cases?

Yes: * There's an enormous amount of very expensive shared state (context cache) which you do not want to duplicate when you can avoid it. * Memory locality is crucially important for performance. * Hardware is extremely over-subscribed. * Hardware is extremely expensive. These factors all make hardware or even traditional memory-space (hypervisor/VM/hardware assisted virtualization) isolation a non-starter for most…

> which forces all isolation to the software layer. This already makes things way harder than they are in commodity SaaS.

Is this not what happens in most SaaS? Isolation at the software layer? I understand there are special agreements, but they seem to be mostly that – no?

> the ratio of usage/sensitivity to maturity is also just poor overall; these are young companies with rapid development and enormous delivery pressure under incredible customer workload requirements, too.

Mh. The talent density in these companies is apparently quite exceptional. Things like customer data separation is something that is obvious and top of mind. I don't see why they would not hire the best to implement these relatively boring/solved things correctly at an architectural level.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#84

Post by andy99 »

gojomo wrote: post #69
andy99 wrote: post #37

Earlier quoted context omitted.

Not sure the relevance of this comment, but normally if someone built a classifier that bad they’d be fired. Anthropic obviously thinks they have some monopoly power they can use to foist garbage on consumers, I think they don’t.

If people are complaining about Anthropic (on an only-vaguely related thread) rather than simply switching to a suitable competitor, then Anthropic clearly has some 'monopoly' power over the specific capabilities the complainer wants from them.

Not to argue the point but that statement isn’t logical, look at all the complaints about restaurants. Publicly complaining about something doesn’t require it be a monopoly.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#85

Post by pseudony »

solenoid0937 wrote: post #64
minhaz23 wrote: post #63

Earlier quoted context omitted.

Curious why you feel that way about Dario?

HN thinks the safety crowd is dumb, and has never seriously engaged with the AI safety space. HN doesn't believe superintelligence will be a thing; while the AI safety crowd believes they are building it. So the decisionmaking of the safety crowd is incomprehensible to HN.

[flagged]

Re: Potential session/cache leakage between workspace instances or consumer accounts

#86

Post by stavros »

Avicebron wrote: post #7

In order Fable 5 has rejected: "Recipe for red-braised pork, I have pork shoulder" "Write up a framework for MCP patterns I can give to claude code" "explain the biomechanics of motion in c. elegans" (I get this one, I mostly did it to test and it's related to my hobby project) Do we get an extra day of functional Fable 5 because it's down?

I asked it how people get blue eyes from their parents and it downgraded me to Opus because of safety.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#87

Post by markasoftware »

pocksuppet wrote: post #74
throwaway260704 wrote: post #57

Using a throwaway account for obvious reasons, but I’m very involved in this space using LLMs from multiple providers. I’m aware of at least two instances in which the intermediate infrastructure “swapped” responses, once impacting Claude models and once impacting GPT models, from two different providers. One gave us a proper postmortem in which their API gateway was incorrectly handling HTTP 100 status codes, puttin…

This attack is called "HTTP desync" or "request smuggling". It's often done intentionally by a client to try and spy on other clients' responses. Every time you multiplex requests from multiple clients onto one upstream connection, you are probably vulnerable to this, because (despite its superficial simplicity) HTTP is just too complex to reliably match the requests and responses to upstream. For example a desync ca…

Very true, this was likely an attack. Worth noting that mr kettle has done a defcon talk nearly every year on some variant of this attack, the most recent one titled "HTTP/1.1 must die" because he rightfully believes that switching to the binary headers of http/2 (specifically in reverse proxy connections to upstream servers) is the only way to systematically prevent these.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#88

Post by jasonjmcghee »

notnmeyer wrote: post #62
repeekad wrote: post #53

Earlier quoted context omitted.

CLAUDE.md, Anthropic is too exclusive and next level to use a standard idiomatic pattern like AGENTS.md

echo “read @AGENTS.md” > CLAUDE.md

Just use a symbolic link

Re: Potential session/cache leakage between workspace instances or consumer accounts

#89

Post by throwatdem12311 »

politician wrote: post #79
minhaz23 wrote: post #63

Earlier quoted context omitted.

Curious why you feel that way about Dario?

Dario quit OpenAI to hype the AI apocalypse for quick cash and attention. Then, he walked right into an obvious crisis with the Pentagon by continuing to try to play both sides of the AGI doom story that even his own AI would've pointed out. Then, after being labelled a supply chain risk, he starts a new roadshow with the newest most dangerous AI model that definitely cannot be released to the public and its safer li…

All of these things have people frothing at the mouth to give up all their data to Anthropic to use their models and to buy in when the IPO eventually happens.

Seems to me Dario is actually a genius. These are all things that I would to make people believe that my “basically the same as the other guy” product is ackshually best thing ever for real. Trust me bro.

The entire bubble is hype and fear mongering. The technical merits of the products are completely irrelevant at this point. Dario is doing exactly what someone that understands this would do and they are winning.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#90

Post by albinowax_ »

markasoftware wrote: post #87
pocksuppet wrote: post #74

Earlier quoted context omitted.

This attack is called "HTTP desync" or "request smuggling". It's often done intentionally by a client to try and spy on other clients' responses. Every time you multiplex requests from multiple clients onto one upstream connection, you are probably vulnerable to this, because (despite its superficial simplicity) HTTP is just too complex to reliably match the requests and responses to upstream. For example a desync ca…

Very true, this was likely an attack. Worth noting that mr kettle has done a defcon talk nearly every year on some variant of this attack, the most recent one titled "HTTP/1.1 must die" because he rightfully believes that switching to the binary headers of http/2 (specifically in reverse proxy connections to upstream servers) is the only way to systematically prevent these.

I’ll be back next month with a load of fresh vectors in “Can AI Do Novel Security Research? Meet the HTTP Terminator”

https://portswigger.net/research/talks?talkId=36

Maybe my last presentation on the topic! Possibly.

Post reply on HN