Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google is making private AI practical with homomorphic encryption

blog.google

Post reply on HN
Print view View original

181–190 of 305 posts

Re: Google is making private AI practical with homomorphic encryption

#181

Post by bevekspldnw »

bitpush wrote: post #101
bevekspldnw wrote: post #67

Earlier quoted context omitted.

You literally don’t know who I am or the roles I had. So unless you can tell me how many steps you were from Kent Walker and what you worked on I’m gonna bet a hell of a lot I know more than you. Edit to clarify my prior point: some of the technology makes it into the product, but the putative data protections do not. Why? Because there is always a work around, and ads legal will approve it every time.

> You literally don’t know who I am or the roles I had. I'm now curious. Who are you?

Saying that would make it unable for me to use HN as certain companies monitor my social media comments.

Re: Google is making private AI practical with homomorphic encryption

#182

Post by bevekspldnw »

antonvs wrote: post #171
bevekspldnw wrote: post #67

Earlier quoted context omitted.

You literally don’t know who I am or the roles I had. So unless you can tell me how many steps you were from Kent Walker and what you worked on I’m gonna bet a hell of a lot I know more than you. Edit to clarify my prior point: some of the technology makes it into the product, but the putative data protections do not. Why? Because there is always a work around, and ads legal will approve it every time.

Given that Google has a single unit that does $76 billion in annual revenue, $100 billion projected, without ads, raises questions about how relevant your knowledge is.

I should have bracketed out GCP as a theoretical application for this. I was referring to the Google consumer extended universe.

Re: Google is making private AI practical with homomorphic encryption

#183

Post by bevekspldnw »

WarmWash wrote: post #135
bevekspldnw wrote: post #67

Earlier quoted context omitted.

You literally don’t know who I am or the roles I had. So unless you can tell me how many steps you were from Kent Walker and what you worked on I’m gonna bet a hell of a lot I know more than you. Edit to clarify my prior point: some of the technology makes it into the product, but the putative data protections do not. Why? Because there is always a work around, and ads legal will approve it every time.

Google is too internally split up to have a single cohesive "maximize ad revenue" mission, especially now that cloud is the new golden goose and the future of search/advertising is hazy. If cloud can sell compute to enterprise that can only use FHE, then they will not give a fuck if the ad goons are disgruntled about it.

You have the CEO and the VPs who have final call when there are disputes below.

Ads doesn’t care until something affects them, GCP doesn’t.

Re: Google is making private AI practical with homomorphic encryption

#184

Post by adastra22 »

odo1242 wrote: post #134
dhx wrote: post #63

Earlier quoted context omitted.

To throw out some real and up-to-date numbers from [1] for FHE at "128-bit security level", to sort 8x 8-bit unsigned integers on the most ordinary of desktop PCs, wait 3 seconds for the result. Want to sort 32x 8-bit unsigned integers instead? Come back 34 seconds later for the result. update: also see [2] for some primitive unsigned 64-bit integer operation benchmarks with the TFHE-rs library (winner in the sorting…

It’s slightly better for LLMs because FHE is really bad at branches (it ends up essentially having to try both branches), making sorts nearly the worst possible thing to try since it’s all branches. In the case of AI most things are just addition and multiplication which can make some things faster since there aren’t as many branches. But we’re still nowhere near viability.

The flip is also true: LLM inference is very nearly maximally optimal for FHE. Thee costly bit, multiplication of a bunch of compressed floating point numbers, translates 1:1. The only bit I’m not sure about is the softmax sampling at the end, but that’s just once per token.

On the other hand, each FHE step is a LOT more costly (e.g. elliptic curve exponentiation) than a vectorized BF8 multiply on GPU.

Unlike the sorting number case, it is probably same complexity. But no hardware support AND a massive slow down constant even if you were comparing apples to apples on hardware.

Disclaimer: I spent a decade working on crypto systems but I am not familiar with TFA’s research.

Re: Google is making private AI practical with homomorphic encryption

#185

Post by surajrmal »

bevekspldnw wrote: post #182
antonvs wrote: post #171

Earlier quoted context omitted.

Given that Google has a single unit that does $76 billion in annual revenue, $100 billion projected, without ads, raises questions about how relevant your knowledge is.

I should have bracketed out GCP as a theoretical application for this. I was referring to the Google consumer extended universe.

Which is also learning how to monetize their offering via cloud instead of ads. So again your expertise here is questionable.

Re: Google is making private AI practical with homomorphic encryption

#186

Post by Eueudhsbsj32 »

siddthesquid wrote: post #132
llleeeoooh wrote: post #120

Earlier quoted context omitted.

this is not entirely true, I think. secure enclaves can provide guarantee such that even the host machine cannot inspect the contents within the VM. so even though the AI model itself needs to see plaintext, all is happening in the enclave which the provider cannot see. the main difference is where the guarantee comes from. for FHE, it comes from math, which we trust. for secure enclave, the guarantee comes from Inte…

- If an AI provider has control over the AI algorithm running in the secure enclave, they can easily have functions that provide them the plaintext through a separate channel. secure enclave does not prevent that - The output can reveal information to the provider, which homomorphic encryption would have protected - Inference is running on GPUs - so its moreso nvidia than amd/intel, but this is just a nit So homomorp…

> they can easily have functions that provide them the plaintext through a separate channel

That's why they provide cryptographic attestation that the open model they're running is exactly what they advertise without any modifications.

That combined with GPU confidential compute should protect your LLM prompt and output.

Re: Google is making private AI practical with homomorphic encryption

#187

Post by epwr »

Eueudhsbsj32 wrote: post #139
delusional wrote: post #131

Earlier quoted context omitted.

There's a much more simple and much more efficient way to do it: Policy and Legal contract.

A contract doesn't protect your data from government searchs, with or without warrant.

Wouldn’t a zero data retention agreement be pretty close to this?

Re: Google is making private AI practical with homomorphic encryption

#189

Post by stephen_cagle »

I see a lot of comments here about how private AI is much cheaper than doing this, and that is currently true.

But I am curious if anyone can think of any use cases for this as it currently stands? For every idea I can think of, I can think of another solution that would probably be a better solution at current slowdown in cost from using homomorphic encryption.

For their two examples...

1) For the "will they click the add" one, how would anyone verify that the model I am uploading of myself is remotely accurate? And can't they learn my characteristics by running them against very specific ads that pinpoint my preferences?

2) For the "Do you have heart disease" one, why not just generating a billion datapoints (or more). The user can download the compressed data of this huge sample of points and then just enter their data on their own side and find the point that is closest (if not an exact match) to their own data?

I'm not pulling apart the idea of homorphic encryption, I'm just curious if anyone can think of a really compelling use case for it given it 10^3 to 10^6 greater compute cost?

Post reply on HN