Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Machine Learning on Encrypted Data Without Decrypting It

juliacomputing.com

Post reply on HN
Print view View original

61–70 of 122 posts

Re: Machine Learning on Encrypted Data Without Decrypting It

#61

Post by garganzol »

theschwa wrote: post #6
brenden2 wrote: post #3

If you can infer information from encrypted data then it's not properly encrypted. Generally you would use a salt that would render this type of analyses useless.

The person who runs the model won’t be able to infer anything. The result of the model is itself encrypted, and would need to be decrypted using the original encryption key.

>would need to be decrypted using the original encryption key

A small correction. Decryption is performed with a decryption (private) key which is kept in secret and only known to the data owner.

In contast, encryption key is public and known to both data owner and "the person who runs the model".

Re: Machine Learning on Encrypted Data Without Decrypting It

#62

Post by mirekrusin »

orbifold wrote: post #41
ChrisRackauckas wrote: post #20

This blog post reminds me of the "Machine Learning Systems are Stuck in a Rut" paper [1], where they mentioned: > It is hard to experiment with front end features like named dimensions, because it is painful to match them to back ends that expect calls to monolithic kernels with fixed layout. On the other hand, there is little incentive to build high quality back ends that support other features, because all the fron…

Julia always seemed great on paper and definitely is a strong candidate for replacing Matlab. But whenever I tried using it, the user experience seemed much more broken than python or c++. It just seems way easier to structure and work on a python + c++ project than it is to structure and work on a Julia project. A moderately sized sane c++ code base compiles and runs faster than whatever gymnastics Julia performs to…

clang, swift and julia are all based on llvm compiler toolchain. Ocaml has it's own compiler.

Re: Machine Learning on Encrypted Data Without Decrypting It

#63

Post by garganzol »

Dylan16807 wrote: post #46
mcguire wrote: post #44

Earlier quoted context omitted.

Quick question: If you have control of the computation instructions (the software for your HE CPU), can arrange for a leak of information from the encrypted data?

The software can only do arithmetic in a fixed pattern, no "if" statements or while loops or anything. Any conditionals have to be faked with arithmetic. So, since the arithmetic is secure (otherwise it wouldn't be HE), and the entire runtime pattern is fixed up front and made of nothing but arithmetic, there's no way to leak anything.

This gives a whole new look to lambda calculus and Lisp concepts that promote to represent everything as a (preferably pure) function.

If the whole logic can be expressed as a pure function without conditionals then it would fully fit into HE.

But what if we want conditionals?

Comparison operators like (a b), etc go out of the question immediately as they would allow to guess the values by a simple binary search.

Equality operation (a == b) seems plausible from the security standpoint as it would not reveal the encrypted value. But there is a challenge in performing that operation because both of its arguments may be encrypted with different randomization. To overcome this, probably some neat trick could be performed but... this is the question of the future.

EDIT: Here is an idea. Some FHE engines have pre-encoded values for some magic numbers like 0, 1 and -1.

What if the equality operation p(a, b) = (a == b) is performed like this:

p(a, b) = (a + (b * -1)) == 0

Does anyone have an intuition regarding the feasibility of the proposed trick in something like Microsoft SEAL? Do both left and right parts of the comparison would have the canonicalized randomness making the equality operation possible?

P.S. On a side note, the sheer existence of an equivalence operation in FHE scheme would decimate its security by allowing to plant bruteforce attacks with a lower guesswork. Not a catastrophe by any means, and some systems would prefer that as a small price to pay for having conditionals in a program.

Re: Machine Learning on Encrypted Data Without Decrypting It

#65

Post by falcolas »

KenoFischer wrote: post #4
falcolas wrote: post #2

Before reading: "I bet they're using homomorphic encryption to expose patterns in the encrypted data" After reading: Yup. It makes sense, so long as your resulting model is run against similarly encrypted data, the same patterns will be there for the ML to identify. Which is, of course, one of the issues with homomorphic encryption.

Just to clarify, homomorphic encryption does not expose patterns. At every point in the computation the ciphertexts are computationally indistinguishable from random. The result of evaluating the ML model will be an encrypted prediction that you then need to send back to whoever encrypted the data (or more precisely whoever has the key - doesn't need to be the same person) so they can decrypt and use the prediction.

So, you're training a statistical model - teaching it to recognize certain patterns - on data that is somehow wholly without patterns? Even with a certain amount of noise in the individual data points, if you're given enough data to train a statistical model to identify traits in the ciphertext you also probably have enough data to break the encryption.

Also, something can be statistically random and still have patterns (see PRNGs, which are statically random (you can't identify the next value from previous values), but there's still a pattern if you know the algorithm and seed).

I'll admit, the promise of homomorphic encryption is pretty amazing, but this particular combination of data and ML seems like a fairly obvious way to leak data. I believe there's a reason that homomorphic encryption has not been broadly accepted as an allowed standard.

EDIT: So, I think I'm missing a practical example. I have a model which I want to train on data homomorphically encrypted with key X, and that model is a very simple "is this a cat". I'm given a whole set of data encrypted with key X that's tagged with "cat" and "not a cat".

Once the model is trained, I can run this on any data encrypted with key X and find out if the data contains a cat (with some degree of accuracy). I have no way of telling information outside the tags provided on the training data, but it still gives me, a person without the encryption key, the ability to identify any feature that's tagged in the training set on any un-tagged production set.

Having seen a large quantity of ML training sets, the tag sets are rarely so limited. There's also often "elephant", "ball", and "dog" tags, even if I'm only being asked to train on cats.

Re: Machine Learning on Encrypted Data Without Decrypting It

#66

Post by [deleted] »

garganzol wrote: post #63
Dylan16807 wrote: post #46

Earlier quoted context omitted.

The software can only do arithmetic in a fixed pattern, no "if" statements or while loops or anything. Any conditionals have to be faked with arithmetic. So, since the arithmetic is secure (otherwise it wouldn't be HE), and the entire runtime pattern is fixed up front and made of nothing but arithmetic, there's no way to leak anything.

This gives a whole new look to lambda calculus and Lisp concepts that promote to represent everything as a (preferably pure) function. If the whole logic can be expressed as a pure function without conditionals then it would fully fit into HE. But what if we want conditionals? Comparison operators like (a b), etc go out of the question immediately as they would allow to guess the values by a simple binary search. Equ…

[deleted]

Re: Machine Learning on Encrypted Data Without Decrypting It

#67

Post by thefifthsetpin »

falcolas wrote: post #65
KenoFischer wrote: post #4

Earlier quoted context omitted.

Just to clarify, homomorphic encryption does not expose patterns. At every point in the computation the ciphertexts are computationally indistinguishable from random. The result of evaluating the ML model will be an encrypted prediction that you then need to send back to whoever encrypted the data (or more precisely whoever has the key - doesn't need to be the same person) so they can decrypt and use the prediction.

So, you're training a statistical model - teaching it to recognize certain patterns - on data that is somehow wholly without patterns? Even with a certain amount of noise in the individual data points, if you're given enough data to train a statistical model to identify traits in the ciphertext you also probably have enough data to break the encryption. Also, something can be statistically random and still have patte…

> homomorphic encryption has not been broadly accepted as an allowed standard.

I get that a lack of broad acceptance is a hard thing to source, but do you have one?

Re: Machine Learning on Encrypted Data Without Decrypting It

#68

Post by KenoFischer »

falcolas wrote: post #65
KenoFischer wrote: post #4

Earlier quoted context omitted.

Just to clarify, homomorphic encryption does not expose patterns. At every point in the computation the ciphertexts are computationally indistinguishable from random. The result of evaluating the ML model will be an encrypted prediction that you then need to send back to whoever encrypted the data (or more precisely whoever has the key - doesn't need to be the same person) so they can decrypt and use the prediction.

So, you're training a statistical model - teaching it to recognize certain patterns - on data that is somehow wholly without patterns? Even with a certain amount of noise in the individual data points, if you're given enough data to train a statistical model to identify traits in the ciphertext you also probably have enough data to break the encryption. Also, something can be statistically random and still have patte…

> Once the model is trained, I can run this on any data encrypted with key X and find out if the data contains a cat (with some degree of accuracy). I have no way of telling information outside the tags provided on the training data, but it still gives me, a person without the encryption key, the ability to identify any feature that's tagged in the training set on any un-tagged production set.

I think you're missing the fact that the predictions come back encrypted, so you don't learn anything unless you know the key. Also, in this particular example the model was trained on unencrypted data, but as discussed below, you can do either.

Re: Machine Learning on Encrypted Data Without Decrypting It

#69

Post by KenoFischer »

winrid wrote: post #64

I'm assuming this uses similar encryption algorithms that you can use to do queries on encrypted data?

I'm assuming you're referring to private information retrieval (https://en.wikipedia.org/wiki/Private_information_retrieval), which is from the same field of research, but may or may not use the same techniques.

Re: Machine Learning on Encrypted Data Without Decrypting It

#70

Post by falcolas »

thefifthsetpin wrote: post #67
falcolas wrote: post #65

Earlier quoted context omitted.

So, you're training a statistical model - teaching it to recognize certain patterns - on data that is somehow wholly without patterns? Even with a certain amount of noise in the individual data points, if you're given enough data to train a statistical model to identify traits in the ciphertext you also probably have enough data to break the encryption. Also, something can be statistically random and still have patte…

> homomorphic encryption has not been broadly accepted as an allowed standard. I get that a lack of broad acceptance is a hard thing to source, but do you have one?

https://csrc.nist.gov/publications/detail/sp/800-111/final is a good starting point (and used by HIPPA, as an aside).

It leads to block cipher standards, AES standards, and so on.

I think it's not unreasonable to say that if it should ever appear as a recommended cipher by NIST, then it can be considered to be broadly accepted.

Post reply on HN