Now do it for tensorflow. Let's say I want to generate AI faces on a small ARM processor.
Transform ML models into native code with zero dependencies
11–20 of 27 posts
Re: Transform ML models into native code with zero dependencies
#12Does anyone know of a similar idea for neural networks? As far as I can tell, you need the entire framework (which requires a heavy, 1.5 GB docker image) to apply a trained model, even though in theory you only need matrix multiplication and a few activation functions.
Related: https://onnx.ai/
Re: Transform ML models into native code with zero dependencies
#13Does anyone know of a similar idea for neural networks? As far as I can tell, you need the entire framework (which requires a heavy, 1.5 GB docker image) to apply a trained model, even though in theory you only need matrix multiplication and a few activation functions. Related: https://onnx.ai/
A similar idea but for neural networks is by using TVM [1]. It's built on top of Halide and LLVM. I haven't tried it myself, but it seems to support deployment to many backends.
Re: Transform ML models into native code with zero dependencies
#14This is pretty cool idea! Would there be any possibility of trying to convert a model to javascript/node? Haven't looked through the source but it looks like the generated code is a essentially the weights from a trained model transformed into a function for the target language.
If your model framework supports it, you can export it to ONNX and there’s JS frameworks that support serving ONNX models.
Re: Transform ML models into native code with zero dependencies
#15You might consider supporting sklearn.preprocessing objects as well to replicate any transformations applied to training data.
Re: Transform ML models into native code with zero dependencies
#16Does anyone know of a similar idea for neural networks? As far as I can tell, you need the entire framework (which requires a heavy, 1.5 GB docker image) to apply a trained model, even though in theory you only need matrix multiplication and a few activation functions. Related: https://onnx.ai/
There’s a project that compiles ONNX models: https://onnc.ai
Re: Transform ML models into native code with zero dependencies
#17Re: Transform ML models into native code with zero dependencies
#18Does anyone know of a similar idea for neural networks? As far as I can tell, you need the entire framework (which requires a heavy, 1.5 GB docker image) to apply a trained model, even though in theory you only need matrix multiplication and a few activation functions. Related: https://onnx.ai/
I wrote keras2cpp https://github.com/pplonski/keras2cpp
It transforms keras + theano models into pure C++, no additional packages. It is not using GPU.
Re: Transform ML models into native code with zero dependencies
#19Now do it for tensorflow. Let's say I want to generate AI faces on a small ARM processor.
Tensorflow is working on tflite, which runs on small ARM processors (with runtime).
And tfcompile is their compile-to-native approach (no runtime).
Re: Transform ML models into native code with zero dependencies
#20There are similar libraries for converting ML models into SQL queries.
However, the important part of most models is not the `estimator.fit(X, y)` line, but all the things that are done to X before fitting or estimating.