What are the runtimes?
[deleted]
Machine Learning on Encrypted Data Without Decrypting It
31–40 of 122 posts
Re: Machine Learning on Encrypted Data Without Decrypting It
#32This sounds like what Microsoft research did with SEAL to produce CryptoNets https://www.microsoft.com/en-us/research/publication/crypton...
Yep, same research setting, though a different network. I don't at the moment remember all the details of CryptoNets, but IIRC they were doing batch size 8192 evaluations (i.e. just using each slot as an independent value and evaluating the code as if on scalars), which allows you to get away without the fancy ciphertext encoding magic that's described in the blog post (at the cost of high latency of course).
Re: Machine Learning on Encrypted Data Without Decrypting It
#33What are the runtimes?
I'm a researcher at Microsoft who's worked a lot on this problem (neural network inference on encrypted data). Take a look at this paper we published at PLDI [1] on our CHET compiler. The speed of the inference depends a lot on the depth of the network. A small 2 convolutional layer model takes a couple of seconds, while a 10 convolutional layer SqueezeNet takes 164 seconds. The reason is that you have to select encryption parameters according to the depth of the computation. Since this paper was published we've by the way improved these numbers ~5X.
That was optimizing for latency and throughput is a something you can separately optimize for. This landmark paper (also from Microsoft) [2] showed inferencing on 2 convolutional layer network at a throughput of 58k images per hour.
[1]: https://www.cs.utexas.edu/~roshan/CHET.pdf [2]: http://proceedings.mlr.press/v48/gilad-bachrach16.pdf
Re: Machine Learning on Encrypted Data Without Decrypting It
#34Earlier quoted context omitted.
Maybe i misunderstood something, but they are not really inferring information. The model is still encrypted, the outsider doesn't know what's going on. Wouldn't salt destroy the homomorphic property?
Homomorphic encryption is malleable[1] in that it can, with enough information, be decrypted without knowing the private keys in some cases. For example, if you can correlate with other data it may be possible to effectively undo the encryption. This is more like anonymization (effectively a one-way hash) than encryption. If you encrypt 2 different values with the same algorithm and key, you will get the same ciphert…
I believe that most HE algorithms (CKKS included) use some random noise, which means you cant infer equality. It's still "malleable", but not to the extent that you assert.
Re: Machine Learning on Encrypted Data Without Decrypting It
#35Earlier quoted context omitted.
Maybe i misunderstood something, but they are not really inferring information. The model is still encrypted, the outsider doesn't know what's going on. Wouldn't salt destroy the homomorphic property?
Homomorphic encryption is malleable[1] in that it can, with enough information, be decrypted without knowing the private keys in some cases. For example, if you can correlate with other data it may be possible to effectively undo the encryption. This is more like anonymization (effectively a one-way hash) than encryption. If you encrypt 2 different values with the same algorithm and key, you will get the same ciphert…
This is wholly untrue. The guarantees HE gives for security of encrypted data is exactly the same as any traditional symmetric or asymmetric crypto system. HE is indeed malleable by design, but doing computation on homomorphically encrypted data gives no information on the contents of the ciphertexts.
>This is more like anonymization (effectively a one-way hash) than encryption. If you encrypt 2 different values with the same algorithm and key, you will get the same ciphertext which reveals information about the original value (i.e., that they are the same).
This is a misunderstanding. Homomorphic encryption schemes are randomized encryptions schemes, which means even if you encrypt the same value twice with the same encryption key you get two different ciphertexts (both look like "random data"). They just have the property that they will decrypt to the same value. This is possible because the space of ciphertexts is much larger than the space of plaintexts.
Re: Machine Learning on Encrypted Data Without Decrypting It
#36Earlier quoted context omitted.
There is no reason to reuse the same key though...
If you used a different key for each datum then you wouldn't be able to do this type of analyses. This analyses depends on the property of the same values producing the same ciphertext, which also mean you're leaking information.
No it doesn’t. The model is trained on unencrypted data. It is then run on encrypted data sent by the client to generate an encrypted classification. The client then uses its decryption key to decrypt the encrypted classification. This process is possible because the model is built to commute with encryption.
Re: Machine Learning on Encrypted Data Without Decrypting It
#37Before reading: "I bet they're using homomorphic encryption to expose patterns in the encrypted data" After reading: Yup. It makes sense, so long as your resulting model is run against similarly encrypted data, the same patterns will be there for the ML to identify. Which is, of course, one of the issues with homomorphic encryption.
This is not how this works. Homomorphic encryption (HE) is best thought of as a software CPU that gives you ways to mutate data (but not read it without the secret key of course). Any computation you run has to be using the instruction set offered by HE: addition, multiplication or rotation. When you do an operation on two ciphertexts, you get a new ciphertext back, but no information on its contents.
Now the way you do machine learning here is by translating your model to use the instructions offered by HE. You've effectively recompiled the model to a new architecture.
If you'd like to read more about machine learning with homomorphic encryption, we published a paper on our CHET compiler [1]. I also talk about this space on a high level in this MSR podcast episode [2].
[1]: https://www.cs.utexas.edu/~roshan/CHET.pdf [2]: https://www.microsoft.com/en-us/research/blog/he-compilers-f...
Re: Machine Learning on Encrypted Data Without Decrypting It
#38This sounds like what Microsoft research did with SEAL to produce CryptoNets https://www.microsoft.com/en-us/research/publication/crypton...
Yep, same research setting, though a different network. I don't at the moment remember all the details of CryptoNets, but IIRC they were doing batch size 8192 evaluations (i.e. just using each slot as an independent value and evaluating the code as if on scalars), which allows you to get away without the fancy ciphertext encoding magic that's described in the blog post (at the cost of high latency of course).
You're right, CryptoNets used a data layout optimized for throughput with a batch size 4096. Since then we've done a lot of work on low latency inference with our CHET compiler [1] and my colleagues with LoLa [2]. It all comes down to the data layouts you use.
[1]: https://www.cs.utexas.edu/~roshan/CHET.pdf [2]: https://arxiv.org/pdf/1812.10659.pdf
Re: Machine Learning on Encrypted Data Without Decrypting It
#39Not sure I totally grok this, but this has been around for a while to use tensorflow on encrypted data [0] [0] https://github.com/tf-encrypted/tf-encrypted
TF Encrypted has focused more on MPC [1]. At least that's what their arXiv paper [2] talks about. It does seem they are also working to integrate Microsoft SEAL for HE.
[1]: https://en.wikipedia.org/wiki/Secure_multi-party_computation [2]: https://arxiv.org/abs/1810.08130
Re: Machine Learning on Encrypted Data Without Decrypting It
#40As somebody with some ML background but no expertise in crypto, is the following ELI~20 summary correct? We take an ML model trained on unencrypted data, use a 'homomorphic evaluation' technique (let's just leave that as magic here) to convert the model operation-by-operation to a model that runs on encrypted data, do a little more crypto magic, and we've solved the business problem described at the beginning of the…
I'm piecing it together too. This is my understanding: the "model" has a set of rules (add, divide, bit shift) and that they do funky encrypted actions (add by making every third bit flip or something). If you run all of these actions in place of what you'd normally do (ie, just add like normal), then return the product, it can be decrypted as a result of the functions ran on it.
I might be wrong though.