Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google is making private AI practical with homomorphic encryption

blog.google

Post reply on HN
Print view View original

271–280 of 305 posts

Re: Google is making private AI practical with homomorphic encryption

#271

Post by sumeno »

bevekspldnw wrote: post #67
luckydata wrote: post #57

Earlier quoted context omitted.

You are very wrong about all of this btw.

You literally don’t know who I am or the roles I had. So unless you can tell me how many steps you were from Kent Walker and what you worked on I’m gonna bet a hell of a lot I know more than you. Edit to clarify my prior point: some of the technology makes it into the product, but the putative data protections do not. Why? Because there is always a work around, and ads legal will approve it every time.

Ads does not have access to GCP data and there is no way legal would approve one

Re: Google is making private AI practical with homomorphic encryption

#272

Post by eventualcomp »

bitpush wrote: post #47
Cider9986 wrote: post #22

This is the same Google that doesn't have e2ee on their password manager by default. Like WTF, it's a password manager.

F. Scott Fitzgerald's test of top-tier intelligence - > Holding two opposing views in the mind means accepting two contradictory ideas at the same time without needing to pick one side or rush to a simple answer I continue to use Apple products because they are top class even though everytime I think of Tim Cook in the Oval Office presenting the gold plaque to the current president, it makes me wanna puke. World isnt…

Why are we citing novelists for appeals to authority on something like intelligence. I thought this might be a figure on developmental psychology to give value to your comment, but it's not even that. Should I be citing George Orwell's doublethink to counter you?

Re: Google is making private AI practical with homomorphic encryption

#273

Post by avmich »

joquarky wrote: post #196
bitpush wrote: post #100

Earlier quoted context omitted.

Autonomous driving was all show until Waymo happened AI research was all show until transformers happened. Quantum research is all show until .. and now this. Part of changing the world involves imagining a changed-world.

Cold fusion and room temperature superconductors (remember LK-99?) were also all show.

You can't prove the theory by providing an agreeing example though.

That for some cases they are still all show doesn't prove everything is all show.

Re: Google is making private AI practical with homomorphic encryption

#274

Post by Cider9986 »

oofbey wrote: post #203
Cider9986 wrote: post #22

This is the same Google that doesn't have e2ee on their password manager by default. Like WTF, it's a password manager.

Former Googler here. E2EE is easy. Nobody gets promoted at Google for solving easy problems. In fact if you set out to solve an easy problem, it looks bad at performance review time.

How is E2EE easy?

Re: Google is making private AI practical with homomorphic encryption

#276

Post by covoeus »

yorwba wrote: post #230
skew-aberration wrote: post #212

Earlier quoted context omitted.

Softmax is once per token per a layer, and growing linearly with context window size (therefore quadratic over full input).

That's softmax dot-product attention. It's quadratic even without fully-homomorphic encryption, but at least it's not inherently branchy, so won't necessarily slow down much more than other floating point operations under encryption. But softmax sampling, where you pick a single output token at the end and feed it back in to generate the next one, is branchy, so you need to do some extra encrypted computation to avoi…

Assuming the client is online, you could offload the costly FHE sampling by handing them the raw logits.

Re: Google is making private AI practical with homomorphic encryption

#277

Post by traceroute66 »

epwr wrote: post #187
Eueudhsbsj32 wrote: post #139

Earlier quoted context omitted.

A contract doesn't protect your data from government searchs, with or without warrant.

Wouldn’t a zero data retention agreement be pretty close to this?

> Wouldn’t a zero data retention agreement be pretty close to this?

1. A ZDR clause is "trust me bro". You have zero way of verifying their pinky-promise.

2. A ZDR clause is still subject to the old-classic "government, court or administrative order" catch-all clause. :)

3. "Even with ZDR enabled, Anthropic may retain data where required by law or to address Usage Policy violations. If a session is flagged for a policy violation, Anthropic may retain the associated inputs and outputs for up to 2 years, consistent with Anthropic’s standard ZDR policy." (I quoted Anthropic, I'm sure all the others have similar).

Re: Google is making private AI practical with homomorphic encryption

#278

Post by odo1242 »

matthewdgreen wrote: post #174
odo1242 wrote: post #134

Earlier quoted context omitted.

It’s slightly better for LLMs because FHE is really bad at branches (it ends up essentially having to try both branches), making sorts nearly the worst possible thing to try since it’s all branches. In the case of AI most things are just addition and multiplication which can make some things faster since there aren’t as many branches. But we’re still nowhere near viability.

I’m genuinely not an expert, but isn’t the beauty of MoE models the fact that we explicitly don’t evaluate every parameter on inference? We evaluate exactly the subset that are needed to evaluate a prompt. Seems like this will bring back data-dependent branches again.

Pretty much, and this does a good job of illustrating the fundamental issue with branching. You could use an encryption scheme that allows the server to determine what MoE expert to load (the simplest would be to have the client decode the value and send it back to the server, though this can sometimes be possible to do without the round trip), but then it’s not fully homeomorphic because the server has some info about the computation that could be used to recover stuff about the original text.

Taking the above point to the extreme, a very simple yet mildly effective “homeomorphic encryption” scheme would be to run the first layer(s) of the ML model on-device, run the majority of the model in cloud, then run the remainder of the model on the device. But then you leak a lot of information that can essentially be used to get back the original text. (Usually in this type of scheme, to defend against this, the provider of cloud services doesn’t have access to the full model, it’s been used before on vision applications involving medical data)

Re: Google is making private AI practical with homomorphic encryption

#279

Post by snovv_crash »

furkanturan wrote: post #242
snovv_crash wrote: post #229

So much inefficiency just to run it on someone else's untrusted hardware. Private AI is already possible today with local open-weight models running on hardware you control. Homomorphic encryption is cool technology, but I'm really not sure what problem it solves.

There are many challenges we need to solve for privacy preserving compute. Legislations, legal matters, key management, HSM like devices that has limited compute capabilities. At Belfort, we are accelerating FHE, besides exploring such uses cases for FHE. Like this one with Google, we also have publishes use cases; https://belfortlabs.com/blog/encrypted-fraud-detection-with-... https://belfortlabs.com/blog/belfort-pa…

The advertising case seems to be solved by local models as well, then the data doesn't even need to be sent to the server.

The Swift one is interesting, but I don't understand why the transaction needs to be encrypted, or more specifically why the model can only run in the encrypted part of the pipeline.

Super cool tech, but I guess I just don't see the application yet.

And wow, I worked with Marc a decade ago, small world.

Re: Google is making private AI practical with homomorphic encryption

#280

Post by stanmancan »

neya wrote: post #270
snovv_crash wrote: post #229

So much inefficiency just to run it on someone else's untrusted hardware. Private AI is already possible today with local open-weight models running on hardware you control. Homomorphic encryption is cool technology, but I'm really not sure what problem it solves.

I bet this would've been ground breaking if this was an announcement from Apple though.

To be fair, Apple and Google are on opposite sides of the "respects user privacy" spectrum.
Post reply on HN