Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fooling around with encrypted reasoning blobs

blog.cryptographyengineering.com

Post reply on HN
Print view View original

31–36 of 36 posts

Re: Fooling around with encrypted reasoning blobs

#31

Post by voxic11 »

boriselec wrote: post #28

Why do reasoning blocks even get encrypted? Reasoning can’t contain information that is more ‘sensitive’ than assistant response. It is annoying to be not able to see reasoning tokens.

They are encrypted to prevent others from training on the reasoning trace. But before anthropic started encrypting the reasoning traces they were signed. The signature was to prevent the user from being able to manipulate the reasoning part of the context because this reasoning part is considered "more trustworthy" by the model. So it would be bad if the user could manipulate the reasoning to convince the model to do something dangerous/against policy. Encryption keeps this property while also preventing the reasoning from being used for training.

Re: Fooling around with encrypted reasoning blobs

#32

Post by bruce343434 »

brookst wrote: post #24
mycall wrote: post #21

Earlier quoted context omitted.

While it seems like a good idea, resending a growing context window is very inefficient and costly. Instance pinning would make a huge efficiency gains but also collapse LLM provider revenue. This is something open models could better solve.

Can you elaborate? How could it be more efficient and bad for revenue? Would it also be bad for profit?

More efficient in terms of bandwidth (not) used. More costly because it has to be stored somewhere instead.

Re: Fooling around with encrypted reasoning blobs

#33

Post by mswphd »

mycall wrote: post #21
glitchc wrote: post #4

Very interesting. The state management is the really insightful find here. I always wondered how these large AI companies managed access for millions of simultaneous users without having to allocate a dedicated LLM instance for each user. Pushing the complete state down to the user after every call makes perfect sense. The LLM itself stays memoryless and ready to respond to an arbitrary prompt. Very nice.

While it seems like a good idea, resending a growing context window is very inefficient and costly. Instance pinning would make a huge efficiency gains but also collapse LLM provider revenue. This is something open models could better solve.

even a max size context window is what, ~1M? iirc tokens are generally part of a vocab of size ~300k. Assume no compression before the encryption (no clue if this is true, but compressing text before encryption can leak info regarding the message, namely how compressible it is), that's \log2 300k ~ 18 bits per token, or ~2 bytes. So each "turn" would involve ~2MB extra in each direction. And again, this is assuming max context.

seems plausibly fine

Re: Fooling around with encrypted reasoning blobs

#35

Post by MagicMoonlight »

Fuck these companies man. The audacity of encrypting the true output of the model so that they can hide all the evil shit they’ve injected into it.

“Remember - the user loves Diet Coke. Subtly insert references to it whenever possible. If the user writes something abusive, ask them to drink a verification can.”

Re: Fooling around with encrypted reasoning blobs

#36

Post by vachina »

geocar wrote: post #5
glitchc wrote: post #4

Very interesting. The state management is the really insightful find here. I always wondered how these large AI companies managed access for millions of simultaneous users without having to allocate a dedicated LLM instance for each user. Pushing the complete state down to the user after every call makes perfect sense. The LLM itself stays memoryless and ready to respond to an arbitrary prompt. Very nice.

N.B. This is exactly how seaside, vba, and even arc[1] do server-side state generally : by encrypting the blob-representing-state and sending to the client to be sent back on future requests (where it will be decrypted and rehydrated). It's an old trick that everyone designing protocols should know, since there are lots of applications beyond AI companies. [1]: As in, pg's lisp: https://arclanguage.github.io/ref/srv.…

Also basically jwt tokens
Post reply on HN