Is there really a need to shrink models? As far as I know, most machine learning models can be very compact, often well under 1 GB. Even high-res vision CNNs aren't anywhere close to being fully-connected. They might have millions of weights, but that's just in the MegaByte range. My understanding in the the real problem is obfuscating the machine learning model. If they decide to put their model on the local memory,…
For a security product deployment of the model over the internet is a significant cost. Hundreds of thousands of machines in an enterprise may need to be updated at once. These machines may not have an easy way to accelerate the evaluation of the model, either, making feature reduction important for CPU work as well.