Fooling around with encrypted reasoning blobs
blog.cryptographyengineering.com
Fooling around with encrypted reasoning blobs
1–10 of 36 posts
Re: Fooling around with encrypted reasoning blobs
#2Very cool idea to use thinking duration (either in tokens or in wall time) as a side-channel!
Re: Fooling around with encrypted reasoning blobs
#3Super cool side channel attack. I tend to agree that it's pretty impractical, but it's such a fun discovery!
Re: Fooling around with encrypted reasoning blobs
#4Very interesting. The state management is the really insightful find here.
I always wondered how these large AI companies managed access for millions of simultaneous users without having to allocate a dedicated LLM instance for each user. Pushing the complete state down to the user after every call makes perfect sense. The LLM itself stays memoryless and ready to respond to an arbitrary prompt. Very nice.
Re: Fooling around with encrypted reasoning blobs
#5Very interesting. The state management is the really insightful find here. I always wondered how these large AI companies managed access for millions of simultaneous users without having to allocate a dedicated LLM instance for each user. Pushing the complete state down to the user after every call makes perfect sense. The LLM itself stays memoryless and ready to respond to an arbitrary prompt. Very nice.
N.B. This is exactly how seaside, vba, and even arc[1] do server-side state generally: by encrypting the blob-representing-state and sending to the client to be sent back on future requests (where it will be decrypted and rehydrated).
It's an old trick that everyone designing protocols should know, since there are lots of applications beyond AI companies.
[1]: As in, pg's lisp: https://arclanguage.github.io/ref/srv.html#:~:text=The%20pre...
Re: Fooling around with encrypted reasoning blobs
#6One possible use for the "replay across accounts": if you can get a reasoning block that jailbreaks the model, you could share that block without sharing how you did it, and others can immediately take advantage of it too.
Re: Fooling around with encrypted reasoning blobs
#7[dead]
Re: Fooling around with encrypted reasoning blobs
#8One possible use for the "replay across accounts": if you can get a reasoning block that jailbreaks the model, you could share that block without sharing how you did it , and others can immediately take advantage of it too.
Not necessarily for the "without sharing" part, but to increase the reliability of the jailbreak. The same prompt isn't guaranteed to return the same result, but combining the internal thinking with the prompt might be a more effective way
Re: Fooling around with encrypted reasoning blobs
#9Very interesting. The state management is the really insightful find here. I always wondered how these large AI companies managed access for millions of simultaneous users without having to allocate a dedicated LLM instance for each user. Pushing the complete state down to the user after every call makes perfect sense. The LLM itself stays memoryless and ready to respond to an arbitrary prompt. Very nice.
the exchange rate between text and its representation in memory is brutal. here's a bit from a recent article:
>An 82 GB footprint in DDR3 on a 2016 Xeon. About 25 GB of weights and 56 GB of KV cache at the full 262K context. The KV cache is larger than the model.
262k tokens is not much at all. with ~5 characters per token, that's only 1.3 MB of plaintext.
Re: Fooling around with encrypted reasoning blobs
#10Awesome write-up. Seems like a great way to play with model responses now that prefill is gone.