Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google is making private AI practical with homomorphic encryption

blog.google

Post reply on HN
Print view View original

171–180 of 305 posts

Re: Google is making private AI practical with homomorphic encryption

#171

Post by antonvs »

bevekspldnw wrote: post #67
luckydata wrote: post #57

Earlier quoted context omitted.

You are very wrong about all of this btw.

You literally don’t know who I am or the roles I had. So unless you can tell me how many steps you were from Kent Walker and what you worked on I’m gonna bet a hell of a lot I know more than you. Edit to clarify my prior point: some of the technology makes it into the product, but the putative data protections do not. Why? Because there is always a work around, and ads legal will approve it every time.

Given that Google has a single unit that does $76 billion in annual revenue, $100 billion projected, without ads, raises questions about how relevant your knowledge is.

Re: Google is making private AI practical with homomorphic encryption

#172

Post by Melatonic »

luckydata wrote: post #110
bevekspldnw wrote: post #67

Earlier quoted context omitted.

You literally don’t know who I am or the roles I had. So unless you can tell me how many steps you were from Kent Walker and what you worked on I’m gonna bet a hell of a lot I know more than you. Edit to clarify my prior point: some of the technology makes it into the product, but the putative data protections do not. Why? Because there is always a work around, and ads legal will approve it every time.

and you don't know who I am. I might have been closer to that work than you would know.

I'm Ron Burgundy ?

Re: Google is making private AI practical with homomorphic encryption

#174

Post by matthewdgreen »

odo1242 wrote: post #134
dhx wrote: post #63

Earlier quoted context omitted.

To throw out some real and up-to-date numbers from [1] for FHE at "128-bit security level", to sort 8x 8-bit unsigned integers on the most ordinary of desktop PCs, wait 3 seconds for the result. Want to sort 32x 8-bit unsigned integers instead? Come back 34 seconds later for the result. update: also see [2] for some primitive unsigned 64-bit integer operation benchmarks with the TFHE-rs library (winner in the sorting…

It’s slightly better for LLMs because FHE is really bad at branches (it ends up essentially having to try both branches), making sorts nearly the worst possible thing to try since it’s all branches. In the case of AI most things are just addition and multiplication which can make some things faster since there aren’t as many branches. But we’re still nowhere near viability.

I’m genuinely not an expert, but isn’t the beauty of MoE models the fact that we explicitly don’t evaluate every parameter on inference? We evaluate exactly the subset that are needed to evaluate a prompt. Seems like this will bring back data-dependent branches again.

Re: Google is making private AI practical with homomorphic encryption

#175

Post by charcircuit »

matthewdgreen wrote: post #174
odo1242 wrote: post #134

Earlier quoted context omitted.

It’s slightly better for LLMs because FHE is really bad at branches (it ends up essentially having to try both branches), making sorts nearly the worst possible thing to try since it’s all branches. In the case of AI most things are just addition and multiplication which can make some things faster since there aren’t as many branches. But we’re still nowhere near viability.

I’m genuinely not an expert, but isn’t the beauty of MoE models the fact that we explicitly don’t evaluate every parameter on inference? We evaluate exactly the subset that are needed to evaluate a prompt. Seems like this will bring back data-dependent branches again.

It would also kill speculative decoding. You would have to run a full inference pass for every token instead of being able to generate multiple tokens with a single pass.

Re: Google is making private AI practical with homomorphic encryption

#176

Post by nl »

bmenrigh wrote: post #31

Has FHE really progressed so far that it's now so efficient that doing computation on an encrypted prompt is feasible? I thought even basic operations like FHE addition were still thousands of times more complex. The only mention in the article I see is: > But while homomorphic encryption has a nontrivial cost overhead, it shifts the capability/privacy trade-off to a question of cost. And the cost of homomorphic encr…

It's not running LLMs.

Re: Google is making private AI practical with homomorphic encryption

#177

Post by speedstyle »

filup wrote: post #53
Ar-Curunir wrote: post #30

Earlier quoted context omitted.

No, that’s not what proper encryption means. Security for encryption means that cipher texts encrypting distinct messages are indistinguishable. This is called IND-CPA, and FHE satisfies this.

How can it possibly pass indcpa. If the model can give me any valuable information about the cipher. Apparently the middle man would know precisely what is contained in the payload.

If you send someone `key xor X`, they can reply with `key xor X xor Y`, and you can recover `X xor Y` – without them having a clue what X is.

Hopefully then it's not too hard to believe you can build an encryption scheme and operations where `dec(enc(X) ⊞ Y) = X + Y` and `dec(enc(X) ⊠ Y) = X × Y`. If these operations can also be composed, i.e. homomorphism, then they're Turing complete! You can turn any function f(X) into a ciphertext version `dec(F(enc(X))) = f(X)`. It can't leak any information about X, because you only ever provided enc(X). For example you transform 'branching' code to something which traverses every path, and paths which do nothing will still appear to randomize the ciphertext, because you can't invent operations that distinguish which path is taken, because it's a secure ciphertext.

Modern homomorphic encryption uses more complex encryption schemes and primitives, to achieve thousands/hundreds rather than millions of times slower than f. Nonetheless it's impossible for the provider to gain a single bit of information about X.

Re: Google is making private AI practical with homomorphic encryption

#178

Post by ameliaquining »

Cider9986 wrote: post #89
ameliaquining wrote: post #77

Earlier quoted context omitted.

Google Password Manager does in fact have E2EE. It's quite straightforward to enable. Your interlocutor is complaining that it's not the default, which I'm quite sure is a deliberate decision made on the grounds that the typical user, the one for whom the default matters because they won't tweak their settings, needs workable account recovery more than they need defense against the specific kinds of security threats…

LastPass, designed for regular users and one of the most popular password managers, had a breach of user vaults. If they hadn't implemented E2EE then it would have been more devastating than it was (crypto wallets were found from vaults with weak master passwords).

Google's security record at avoiding that kind of breach is just about the best in the industry, and their system that handles custody of password manager secrets is designed to withstand even a compromise of their production infrastructure (https://security.googleblog.com/2022/10/SecurityofPasskeysin...). I would advise almost all users to worry more about getting locked out of their password database than about that. Of course, I would also advise almost all users not to self-custody cryptocurrency.

Re: Google is making private AI practical with homomorphic encryption

#180

Post by whyage »

Eueudhsbsj32 wrote: post #90
meindnoch wrote: post #80

Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The most private AI is the one running on my own hardware, not in some giant data center.

Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/

Check out opaque.co
Post reply on HN