Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Potential session/cache leakage between workspace instances or consumer accounts

github.com

Post reply on HN
Print view View original

131–140 of 151 posts

Re: Potential session/cache leakage between workspace instances or consumer accounts

#131

Post by nkrisc »

HumanOstrich wrote: post #35
Avicebron wrote: post #7

In order Fable 5 has rejected: "Recipe for red-braised pork, I have pork shoulder" "Write up a framework for MCP patterns I can give to claude code" "explain the biomechanics of motion in c. elegans" (I get this one, I mostly did it to test and it's related to my hobby project) Do we get an extra day of functional Fable 5 because it's down?

What does this have to do with anything? Who are you talking to? This is Hacker News, not Anthropic support.

I think they’re just posing the question to other users reading these comments.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#133

Post by ShinyLeftPad »

Aurornis wrote: post #118
bix6 wrote: post #20

So the options are this amazing tech is so stupid it just randomly brings up Minecraft or it’s got a major security issue?

The person had “minecraft.py” in their context and the session context was very long. Having an LLM session with very long context occasionally go off on a tangent is not uncommon. The people who expect absolute perfection out of every LLM interaction see this as some total indictment of the entire technology, but the people who use these tools daily have learned to treat the output as partially stochastic and to avo…

TLDR: the first one

> this amazing tech is so stupid it just randomly brings up Minecraft or it’s got a major security issue

You can sugarcoat it but that's what it is. It's not slightly wrong like a junior engineer or weird like a junior engineer on LSD, it becomes like "your junior engineer suffered a stroke or sudden onset dementia completely forgetting the entire point". one trigger word and that's it we're building Minecraft castles now.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#136

Post by solenoid0937 »

superb_dev wrote: post #119
cyral wrote: post #111

Earlier quoted context omitted.

To add to this, they should look at the Fable system card. It's 317 pages and it's clear how serious they are taking AI safety.

Page count is not a measure of how seriously something is being taken when you can easily generate pages and pages of slop

You'd have a point if the report was pages of slop, but it clearly is not.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#138

Post by 27183 »

weitendorf wrote: post #98
27183 wrote: post #28

Earlier quoted context omitted.

I suspect what GP is getting at is there will be a strong incentive to implement some structural sharing across tenants to avoid redundantly storing the same tokens over and over. At least I'd be tempted to do this if I was working with a very precious, constrained resource (e.g. VRAM). Doing this correctly seems.. very difficult. [edit] To answer your question directly: the probability that the entire cache is ident…

Agree with this and I have been thinking about it recently as well. I think you could implement a cord-like vocabulary to identify large duplicated substrings for exact deduplication and pairwise correlations or vocabulary profiles/small classifiers for forward-looking or speculative deduplications. A clear example is the GPL license, it’s a large substring you might encounter often and highly likely to be accompanie…

Given the size of the datacenter class GPUs they're running these models on, don't they need to be processing multiple tenants concurrently per GPU to extract the full potential of the hardware?

I agree, shuffling the data between the CPU and GPU is itself fraught with peril. It's all the hairiest distributed systems problems combined with the sketchiest memory safety issues all in one place.

Re: Potential session/cache leakage between workspace instances or consumer accounts

#140

Post by root_axis »

solenoid0937 wrote: post #64
minhaz23 wrote: post #63

Earlier quoted context omitted.

Curious why you feel that way about Dario?

HN thinks the safety crowd is dumb, and has never seriously engaged with the AI safety space. HN doesn't believe superintelligence will be a thing; while the AI safety crowd believes they are building it. So the decisionmaking of the safety crowd is incomprehensible to HN.

> the AI safety crowd believes they are building it

Stated without a hint of irony.

Post reply on HN