Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google is making private AI practical with homomorphic encryption

blog.google

Post reply on HN
Print view View original

221–230 of 305 posts

Re: Google is making private AI practical with homomorphic encryption

#221

Post by hnaccount_rng »

filup wrote: post #66
drdeca wrote: post #61

Earlier quoted context omitted.

Here’s a very simple one-time-pad style construction for homeomorphic (but not fully homeomorphic) encryption. Suppose the plaintext, ciphertext, and key, are each a natural number modulo 5. The key is selected uniformly at random. The ciphertext is obtained by adding the key to the plaintext (and as a result is also uniform random). Then the ciphertext is sent to the server. The server only has the ciphertext, which…

What does that have to do with incpa? I send this server a question encrypted. It stays encrypted yet the server distinguished my ciphertext enough to produce a coherent reply. That means there was enough information in the ciphertext for the server to perform the inference, even if it was obscured to where it looks indistinguishable, it apparently wasn't. This doesn't remove trust from the equation, it puts the trus…

I think the term information is misleading here. In any encrypted message there is the original information, it must be because we are able to decrypt it. So from an information theory point of view that the server can "distinguished my ciphertext enough to produce a coherent reply" is always possible. The whole point of encryption is that we have Math that allows the identification of that information only under presence of the (or at least some kind of) secret.

The price that HE is paying is essentially "have to run every possible contributing computation", which means it has to scale extremely badly for anything real world complex

Re: Google is making private AI practical with homomorphic encryption

#222

Post by petters »

sabretooth1405 wrote: post #16

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

Only 1000x overhead would make some image classification tasks go from 1ms to 1s. That’s viable for some applications!

Re: Google is making private AI practical with homomorphic encryption

#225

Post by xg15 »

fragmede wrote: post #220
lupire wrote: post #161

Earlier quoted context omitted.

Why on earth do you need a backend for this? The backend only exists because it leaks the data.

because you lose your phone and don't have access to the account anymore.

Then the app can make a bog-standard encrypted-at-rest backup to somewhere and make all the computations on the device on the cleartext data.

I don't see the need to do computations on the encrypted data here, which is what FHE would provide in addition to traditional encryption.

> and don't have access to the account anymore.

This would be trouble with or without FHE. Even if the backend wouldn't need to decrypt the data, the user will - so as soon as you actually want to show something in the app, you have the same key management problems as without FHE.

Re: Google is making private AI practical with homomorphic encryption

#226

Post by akoboldfrying »

fedpost wrote: post #200
never_inline wrote: post #188

I think you folks are reading too much into it. I think the people working on FHE need to publish an AI-oriented pitch to retain funding from AI-pilled execs. Must be the same case with the golang post few days ago.

Yeah I don't think they actually want this. There's far too much deeply personal and useful data being freely feed to the companies running these models. People are giving them everything . Their thoughts, feelings, intentions, company IP... It's a fucking gold mine. There's just too much value in retraining on that data and even if there wasn't, the feds lean so heavily on Google search histories to get convictions…

When McDonalds introduced salads to their menu, purchases shot up -- but ~no one actually bought the salads. They just felt better about buying the same old unhealthy food from McDonalds when they knew that healthier options were available that they could potentially buy.

This is the same kind of thing. The crazy overhead of FHE means it's not practical for almost any kind of real world computational work. But having it available at all is undeniably technically impressive, and some of that sense of technical refinement bleeds over into your opinion of Google as a whole.

Incidentally, being a business client of Google's who actually uses this (or, say, quantum computing) for $BUSINESS_FEATURE would garner the same kind of positive attention, independently of whether it's technically a good idea.

Finally, if the overheads come down to the level where they are still significant but not insane, GCP is in a great position.

Re: Google is making private AI practical with homomorphic encryption

#227

Post by xg15 »

HlessClaudesman wrote: post #223

"hotword" detection, is this basically sniffing encrypted data for words the authorities don't want you to speak? Seems counterproductive to their stated aim of privacy.

Not even just by the authorities. This sounds like the wet dream of Google itself or any other data mining companies. You can just ship the most sensitive data around and extract signals to your hearts content - and even if it would be protected by law, you're good because hey, it's always encrypted, and the signals, ad assignments and profile markers are encrypted too!

This is what I don't like about all this "I'm not touching you" privacy tech, also including the new trend of nonvoluntary on-device processing.

They pretend "privacy" was a primary goal of it's own, completely detached from any reasons why you would want to have it.

Then they go on gloriously holding up the flag of privacy, while at the same time enabling all the things that you wanted to have privacy against in the first place.

Re: Google is making private AI practical with homomorphic encryption

#228

Post by xg15 »

fedpost wrote: post #200
never_inline wrote: post #188

I think you folks are reading too much into it. I think the people working on FHE need to publish an AI-oriented pitch to retain funding from AI-pilled execs. Must be the same case with the golang post few days ago.

Yeah I don't think they actually want this. There's far too much deeply personal and useful data being freely feed to the companies running these models. People are giving them everything . Their thoughts, feelings, intentions, company IP... It's a fucking gold mine. There's just too much value in retraining on that data and even if there wasn't, the feds lean so heavily on Google search histories to get convictions…

Well, wait until there is the first training algorithm that supports FDE. We promise we won't ever decrypt your data, we'll just train our models with it...

Re: Google is making private AI practical with homomorphic encryption

#229

Post by snovv_crash »

So much inefficiency just to run it on someone else's untrusted hardware. Private AI is already possible today with local open-weight models running on hardware you control.

Homomorphic encryption is cool technology, but I'm really not sure what problem it solves.

Re: Google is making private AI practical with homomorphic encryption

#230

Post by yorwba »

skew-aberration wrote: post #212
adastra22 wrote: post #184

Earlier quoted context omitted.

The flip is also true: LLM inference is very nearly maximally optimal for FHE. Thee costly bit, multiplication of a bunch of compressed floating point numbers, translates 1:1. The only bit I’m not sure about is the softmax sampling at the end, but that’s just once per token. On the other hand, each FHE step is a LOT more costly (e.g. elliptic curve exponentiation) than a vectorized BF8 multiply on GPU. Unlike the sor…

Softmax is once per token per a layer, and growing linearly with context window size (therefore quadratic over full input).

That's softmax dot-product attention. It's quadratic even without fully-homomorphic encryption, but at least it's not inherently branchy, so won't necessarily slow down much more than other floating point operations under encryption.

But softmax sampling, where you pick a single output token at the end and feed it back in to generate the next one, is branchy, so you need to do some extra encrypted computation to avoid leaking which token was sampled.

Post reply on HN