Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google is making private AI practical with homomorphic encryption

blog.google

Post reply on HN
Print view View original

231–240 of 305 posts

Re: Google is making private AI practical with homomorphic encryption

#231

Post by adgjlsfhk1 »

amelius wrote: post #143
meindnoch wrote: post #80

Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The most private AI is the one running on my own hardware, not in some giant data center.

> The most private AI is the one running on my own hardware, not in some giant data center. I want that too, but you gotta ask yourself the question how efficient that is compared to running it in a datacenter shared with everybody else.

it's more efficient than paying 1000x for fhe

Re: Google is making private AI practical with homomorphic encryption

#232

Post by furkanturan »

petters wrote: post #222
sabretooth1405 wrote: post #16

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

Only 1000x overhead would make some image classification tasks go from 1ms to 1s. That’s viable for some applications!

Exactly. That is what we have today at Belfort; not enough for making all AI work privacy preserving, but fast enough for many applications, where otherwise unencrypted compute is not acceptable.

Re: Google is making private AI practical with homomorphic encryption

#233

Post by furkanturan »

jacquesm wrote: post #99
abetusk wrote: post #49

Earlier quoted context omitted.

1000x slowdown is bad but not a complete deal breaker. Do you have a sense of what a reasonable achievable factor is? Do you have sense for how long before we get to that achievable factor?

It's a ridiculous waste of energy, just use local compute.

Local compute is preferable where possible. There are cases where computation needs to be performed remotely. For example, when collecting data from remote entities while preserving privacy by allowing each entity to retain ownership of the encryption keys used to protect its data. At Belfort, we are exploring such applications, such as

- https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc... - https://belfortlabs.com/blog/encrypted-fraud-detection-with-...

Re: Google is making private AI practical with homomorphic encryption

#234

Post by furkanturan »

Fordec wrote: post #96
sabretooth1405 wrote: post #16

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

The primary path to speed ups appear to be in custom ASICs by startups like Niobium. Combined with the recent Taalas acquisition by AMD, I think I see where this is going. But yeah, for hot path traffic it's probably going to be swamped by the input data rate. But I expected identity tables and cached lookup data will need to be a core component so duplicate checks is avoided in every way available.

Count Belfort too. In addition to our GPU acceleration efforts, we have ongoing ASIC initiatives to further accelerate encrypted compute.

Re: Google is making private AI practical with homomorphic encryption

#235

Post by alpaca9 »

bevekspldnw wrote: post #56
sabretooth1405 wrote: post #16

My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.

Commercially viable for Google boils down to can they attribute ads behaviors to it or not. Then there’s a second tier of things that just make those wheels turn and if they do or don’t make ads revenue is nominally immaterial. The teams doing this stuff at Google are purely for show, none of this makes it into any real products. There’s the narrow exception of stuff like gboard, that does use privacy preserving ML/f…

Sometimes I feel like 75% of their engineers are working on stuff that is purely for show. If their product doesn't generate enough profit very short term they stop it immediately. They have hunderds of cancelled projects, and those are only the ones which have existed. I wonder how many things they made internally and killed before ever showing it publicly.

Re: Google is making private AI practical with homomorphic encryption

#236

Post by inigyou »

XorNot wrote: post #163
randomImmigrant wrote: post #157

Earlier quoted context omitted.

This seems a fine tradeoff to me, depending on the context. There are datasets and operations on them where speed being sacrificed for privacy/security seems appropriate. Ideally, give me a dial, to ask for encrypted intelligence when I need it. Kind of like a private chat, but with deeper privacy protections.

Except that's for pathetically small datasets. What real datasets exist where this would be a worthwhile trade off versus simply owning the hardware? The numbers are so bad that underpowered local hardware would still beat it.

yep I can beat 0.0008 tokens/s on GLM-5.2 on my CPU

Re: Google is making private AI practical with homomorphic encryption

#237

Post by firtoz »

snovv_crash wrote: post #229

So much inefficiency just to run it on someone else's untrusted hardware. Private AI is already possible today with local open-weight models running on hardware you control. Homomorphic encryption is cool technology, but I'm really not sure what problem it solves.

It's so that they can sell more things to enterprise customers

Re: Google is making private AI practical with homomorphic encryption

#238

Post by furkanturan »

oulipo wrote: post #27

Zama.ai is also a player in this space

Yes, it is. But it’s important to recognize that there are different FHE schemes. Zama has invested heavily in TFHE and achieved impressive results. However, time has shown that TFHE is not the best fit for encrypted AI workloads, with CKKS and related schemes gaining an advantage with mathematical and algorithmic advances continue. At Belfort we have foot on both sides; FPGA acceleration for integrating to Zama's TFHE-rs, and GPU acceleration for CKKS.

Re: Google is making private AI practical with homomorphic encryption

#239

Post by jacquesm »

furkanturan wrote: post #233
jacquesm wrote: post #99

Earlier quoted context omitted.

It's a ridiculous waste of energy, just use local compute.

Local compute is preferable where possible. There are cases where computation needs to be performed remotely. For example, when collecting data from remote entities while preserving privacy by allowing each entity to retain ownership of the encryption keys used to protect its data. At Belfort, we are exploring such applications, such as - https://belfortlabs.com/blog/belfort-partners-with-lg-on-enc... - https://belfo…

For me it is the simplest reasons of all: privacy / confidentiality. There is absolutely no way any of this data leaves my systems.

Re: Google is making private AI practical with homomorphic encryption

#240

Post by Aozora7 »

akoboldfrying wrote: post #226
fedpost wrote: post #200

Earlier quoted context omitted.

Yeah I don't think they actually want this. There's far too much deeply personal and useful data being freely feed to the companies running these models. People are giving them everything . Their thoughts, feelings, intentions, company IP... It's a fucking gold mine. There's just too much value in retraining on that data and even if there wasn't, the feds lean so heavily on Google search histories to get convictions…

When McDonalds introduced salads to their menu, purchases shot up -- but ~no one actually bought the salads. They just felt better about buying the same old unhealthy food from McDonalds when they knew that healthier options were available that they could potentially buy. This is the same kind of thing. The crazy overhead of FHE means it's not practical for almost any kind of real world computational work. But having…

I'll have you know I started picking salads instead of fries in combo meals the moment they became available.

Not that I'm a frequent customer, either before or after.

Post reply on HN