Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google is making private AI practical with homomorphic encryption

blog.google

Post reply on HN
Print view View original

131–140 of 305 posts

Re: Google is making private AI practical with homomorphic encryption

#131

Post by delusional »

Eueudhsbsj32 wrote: post #90
meindnoch wrote: post #80

Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The most private AI is the one running on my own hardware, not in some giant data center.

Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/

There's a much more simple and much more efficient way to do it: Policy and Legal contract.

Re: Google is making private AI practical with homomorphic encryption

#132

Post by siddthesquid »

llleeeoooh wrote: post #120
siddthesquid wrote: post #106

Earlier quoted context omitted.

If I used regular encryption to send my credit card information to an AI with fraud detection, the provider still needs to decrypt that data on their side at some point before it goes into the AI. Using this other encryption, the provider has neither need nor capability to decrypt it on their end, so the user gets extra security.

this is not entirely true, I think. secure enclaves can provide guarantee such that even the host machine cannot inspect the contents within the VM. so even though the AI model itself needs to see plaintext, all is happening in the enclave which the provider cannot see. the main difference is where the guarantee comes from. for FHE, it comes from math, which we trust. for secure enclave, the guarantee comes from Inte…

- If an AI provider has control over the AI algorithm running in the secure enclave, they can easily have functions that provide them the plaintext through a separate channel. secure enclave does not prevent that

- The output can reveal information to the provider, which homomorphic encryption would have protected

- Inference is running on GPUs - so its moreso nvidia than amd/intel, but this is just a nit

So homomorphic encryption exists so the user doesn't need to do work to figure out if the provider could be adversarial.

Re: Google is making private AI practical with homomorphic encryption

#133

Post by joshspankit »

tbenst wrote: post #126
dhx wrote: post #63

Earlier quoted context omitted.

To throw out some real and up-to-date numbers from [1] for FHE at "128-bit security level", to sort 8x 8-bit unsigned integers on the most ordinary of desktop PCs, wait 3 seconds for the result. Want to sort 32x 8-bit unsigned integers instead? Come back 34 seconds later for the result. update: also see [2] for some primitive unsigned 64-bit integer operation benchmarks with the TFHE-rs library (winner in the sorting…

That is sobering for sure, I wonder what the theoretical bounds are on what is possible if known. Would be such a dream to use a Frontier LLM one day with homomorphic encryption, but this sounds wildly implausible based on where things are today.

We’re not even close to the limits of AI optimization so finding the theoretical bounds is going to have to wait

Re: Google is making private AI practical with homomorphic encryption

#134

Post by odo1242 »

dhx wrote: post #63
sabretooth1405 wrote: post #16

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

To throw out some real and up-to-date numbers from [1] for FHE at "128-bit security level", to sort 8x 8-bit unsigned integers on the most ordinary of desktop PCs, wait 3 seconds for the result. Want to sort 32x 8-bit unsigned integers instead? Come back 34 seconds later for the result. update: also see [2] for some primitive unsigned 64-bit integer operation benchmarks with the TFHE-rs library (winner in the sorting…

It’s slightly better for LLMs because FHE is really bad at branches (it ends up essentially having to try both branches), making sorts nearly the worst possible thing to try since it’s all branches. In the case of AI most things are just addition and multiplication which can make some things faster since there aren’t as many branches. But we’re still nowhere near viability.

Re: Google is making private AI practical with homomorphic encryption

#135

Post by WarmWash »

bevekspldnw wrote: post #67
luckydata wrote: post #57

Earlier quoted context omitted.

You are very wrong about all of this btw.

You literally don’t know who I am or the roles I had. So unless you can tell me how many steps you were from Kent Walker and what you worked on I’m gonna bet a hell of a lot I know more than you. Edit to clarify my prior point: some of the technology makes it into the product, but the putative data protections do not. Why? Because there is always a work around, and ads legal will approve it every time.

Google is too internally split up to have a single cohesive "maximize ad revenue" mission, especially now that cloud is the new golden goose and the future of search/advertising is hazy.

If cloud can sell compute to enterprise that can only use FHE, then they will not give a fuck if the ad goons are disgruntled about it.

Re: Google is making private AI practical with homomorphic encryption

#136

Post by glaslong »

fragmede wrote: post #130
sabretooth1405 wrote: post #16

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

The one that I'm waiting for is a women's period tracking app that uses FHE on the backend to be fully private.

Saving your comment for the Weekend Project idea backlog, if you don't mind :)

Re: Google is making private AI practical with homomorphic encryption

#137

Post by glaslong »

u1hcw9nx wrote: post #51
sabretooth1405 wrote: post #16

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

That's the reason for HEIR like optimization and parameter selection. It narrows the 10^3 - 10^6 penalty to 10x - 100x.

Thoughts on whether HE could be further hardware accelerated?

Re: Google is making private AI practical with homomorphic encryption

#138

Post by garbagewoman »

tomhow wrote: post #127
michaelmrose wrote: post #94

Earlier quoted context omitted.

[flagged]

> This is generally the test of extreme stupidity because those so adept rarely realize there is even a contradiction because they lack in critical thinking. > You can separate Cook is a garbage person We've banned this account. We've asked you politely, several times over the years to observe the guidelines. In recent times you've been acting more than ever like they don't apply to you. They do apply to you just as…

Ehh, “dumber than a box of rocks” or similar is more hyperbole than dehumanising speech. Dehumanising speech is calling someone something like a cockroach or a rat. Bit of an overreaction here mate

Re: Google is making private AI practical with homomorphic encryption

#139

Post by Eueudhsbsj32 »

delusional wrote: post #131
Eueudhsbsj32 wrote: post #90

Earlier quoted context omitted.

Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/

There's a much more simple and much more efficient way to do it: Policy and Legal contract.

A contract doesn't protect your data from government searchs, with or without warrant.

Re: Google is making private AI practical with homomorphic encryption

#140

Post by tomhow »

garbagewoman wrote: post #138
tomhow wrote: post #127

Earlier quoted context omitted.

> This is generally the test of extreme stupidity because those so adept rarely realize there is even a contradiction because they lack in critical thinking. > You can separate Cook is a garbage person We've banned this account. We've asked you politely, several times over the years to observe the guidelines. In recent times you've been acting more than ever like they don't apply to you. They do apply to you just as…

Ehh, “dumber than a box of rocks” or similar is more hyperbole than dehumanising speech. Dehumanising speech is calling someone something like a cockroach or a rat. Bit of an overreaction here mate

Sure, in isolation, you can dismiss that kind of language as hyperbole and let it pass. But we're talking about a pattern of rhetorical style and substance over several years that they have been warned about several times, but with no evidence of any intent to reform. HN is not intended for this style of discussion, and people who continue to engage it are driving away people who do want to use HN as intended.
Post reply on HN