Homomorphically encrypted CIFAR-10 inference in 200ms • Hacker News Board

Re: Homomorphically encrypted CIFAR-10 inference in 200ms

#41

Post by wocc » Sat, Jul 18, 2026, 5:59 PM UTC

j2kun wrote: post #17

rcr-anti wrote: post #14

Looked at the network logs and the JS, did some testing, there's a caveat here. For an encryption demo you might expect your secrets to be generated locally, they do the compute on something they can't read, you compare their results to your original plaintext; (imo at least) the point would be that it isn't physically possible for them to cheat. Here, you literally download client_secret.bin from their server, so th…

IMO a big unanswered question in FHE (which is my field so I care about the answer) is key management. If you have a service with lots of different users, and each user requires 100 MiB or more in evaluation key material, simply loading that key material from disk onto the GPU for each user's request adds a nontrivial overhead on the order of magnitude of the FHE computation itself. Not to mention the storage cost (f…

You're identifying one of the (many) underappreciated cost in the RLWE FHE stack. The evaluation key problem doesn't scale, and no amount of hardware acceleration fixes a per-user storage and loading overhead that grows with the security parameter.

Re: Homomorphically encrypted CIFAR-10 inference in 200ms

#42

Post by wocc » Sat, Jul 18, 2026, 6:01 PM UTC

Pro7ech wrote: post #37

jszymborski wrote: post #5

Big if true! One of the caveats here is that the file size seems to balloon 341 times.

This is specific to the current state of the demo. Additional techniques can reduce fresh encryptions to only a few times the plaintext size, but they have not been deployed yet.

[dead]

Re: Homomorphically encrypted CIFAR-10 inference in 200ms

#43

Post by wocc » Sat, Jul 18, 2026, 6:21 PM UTC

j2kun wrote: post #17

rcr-anti wrote: post #14

Looked at the network logs and the JS, did some testing, there's a caveat here. For an encryption demo you might expect your secrets to be generated locally, they do the compute on something they can't read, you compare their results to your original plaintext; (imo at least) the point would be that it isn't physically possible for them to cheat. Here, you literally download client_secret.bin from their server, so th…

IMO a big unanswered question in FHE (which is my field so I care about the answer) is key management. If you have a service with lots of different users, and each user requires 100 MiB or more in evaluation key material, simply loading that key material from disk onto the GPU for each user's request adds a nontrivial overhead on the order of magnitude of the FHE computation itself. Not to mention the storage cost (f…

agreed. The omission of what bit security they are using is of note and they Key missing component of the overhead equation. Private key is better :)

Re: Homomorphically encrypted CIFAR-10 inference in 200ms

#44

Post by smalltorch » Sat, Jul 18, 2026, 6:28 PM UTC

binary132 wrote: post #33

smalltorch wrote: post #7

Earlier quoted context omitted.

Ah, that sounds impossible. Good luck, sir...Also, who wants this if you can just run a local model?

maybe it could be running a much more powerful system than you have access to

I can't tell if your being sarcastic but if you can't prove fhe within ( and I'll be generous) a few 1000 lines of code. Bologna.

Re: Homomorphically encrypted CIFAR-10 inference in 200ms

#45

Post by j2kun » Mon, Jul 20, 2026, 1:11 AM UTC

wocc wrote: post #41

j2kun wrote: post #17

Earlier quoted context omitted.

IMO a big unanswered question in FHE (which is my field so I care about the answer) is key management. If you have a service with lots of different users, and each user requires 100 MiB or more in evaluation key material, simply loading that key material from disk onto the GPU for each user's request adds a nontrivial overhead on the order of magnitude of the FHE computation itself. Not to mention the storage cost (f…

You're identifying one of the (many) underappreciated cost in the RLWE FHE stack. The evaluation key problem doesn't scale, and no amount of hardware acceleration fixes a per-user storage and loading overhead that grows with the security parameter.

I think one counterpoint here is that you can still get value in using FHE in settings that don't involve scaling the number of users (e.g. B2B, public/private collab).