A homomorphic encryption blogpost without any mention of the computational overhead...
Google is making private AI practical with homomorphic encryption
211–220 of 305 posts
Re: Google is making private AI practical with homomorphic encryption
#212Earlier quoted context omitted.
It’s slightly better for LLMs because FHE is really bad at branches (it ends up essentially having to try both branches), making sorts nearly the worst possible thing to try since it’s all branches. In the case of AI most things are just addition and multiplication which can make some things faster since there aren’t as many branches. But we’re still nowhere near viability.
The flip is also true: LLM inference is very nearly maximally optimal for FHE. Thee costly bit, multiplication of a bunch of compressed floating point numbers, translates 1:1. The only bit I’m not sure about is the softmax sampling at the end, but that’s just once per token. On the other hand, each FHE step is a LOT more costly (e.g. elliptic curve exponentiation) than a vectorized BF8 multiply on GPU. Unlike the sor…
Softmax is once per token per a layer, and growing linearly with context window size (therefore quadratic over full input).
Re: Google is making private AI practical with homomorphic encryption
#213Does this also solve provable compute as a side effect? Since the compute is fully encrypted end-to-end, the host wouldn’t be able to modify the program and still give valid results? So a caller could run a program on untrusted hardware and trust that the hardware actually ran the program?
No, with FHE you don't know what program is running on the data, you just know that it can't access your data.
Re: Google is making private AI practical with homomorphic encryption
#214Earlier quoted context omitted.
To what end?
Biometric authentication, credential checking, blind auctions, threat matching (search zero-day signatures without revealing the vectors), private dna matching or running queries of medical symptoms and health records against databases. I think the most immediately useful is checking if a username/password hash appears in a leaked database without revealing which account or password hash is being checked.
I do like that list and thank you for the explanation.
I still worry about how an ad company is going to approach this. Each of these advancements could be used to put ads in more places.
Re: Google is making private AI practical with homomorphic encryption
#215My master's thesis is on a topic in this field (Privacy Preserving ML) and from my understanding HE and other techniques have very high overheads(~10^3) on inference tasks and thus aren't very commercially viable.
It might still be useful for classification usecases
Re: Google is making private AI practical with homomorphic encryption
#216gaslighting people by pretending that encrypting reasoning traces between agent and client is a win for "private AI"... the title should be rewritten as it is plainly conceding -- Closed AI providers use encrypted reasoning traces to hide what the model is doing to come to a conclusion. This is not scientific progress, this is molopoly protectionism. No person using these models wants the reasoning traces hidden from them, it prevents the user from learning how conclusions about a question are derived.
Re: Google is making private AI practical with homomorphic encryption
#217Great, private AI, at the cost of >1000x the resource usage. Because apparently AI companies weren't already using quite enough energy to cook the planet. The most private AI is the one running on my own hardware, not in some giant data center.
Aren't there already much more efficient ways to make inference private? Using regular encryption and secure enclaves, there are already providers that are roughly 2x the cost of normal providers. For example, https://tinfoil.sh/
Yes, by running models locally
Re: Google is making private AI practical with homomorphic encryption
#218Maybe I'm not understanding this, but how is it that you can know enough about the data to process it without undermining the fundamental concept of encryption? Isn't encrypted data supposed to be just random noise without the key? The more you know about the underlying data the easier it gets to decrypt? Does this mean someone can just steal your encrypted data and use that to steal your identity without even needin…
this isnt about the progress of FHE, this is about models hiding reasoning traces (i.e. "thinking") from their paying customers because they dont want them knowing how question A got to answer B.
Re: Google is making private AI practical with homomorphic encryption
#219I‘m new to this, but mildly skeptical. In complex agentic engineering, there are lots of intermediate steps that depend on each other. Even if you disregard the significant encryption cost, this makes it impractical. So seems like this will stay a niche.
Re: Google is making private AI practical with homomorphic encryption
#220Earlier quoted context omitted.
The one that I'm waiting for is a women's period tracking app that uses FHE on the backend to be fully private.
Why on earth do you need a backend for this? The backend only exists because it leaks the data.
because you lose your phone and don't have access to the account anymore.