[deleted]
Leaf: machine intelligence framework in Rust
11–20 of 20 posts
Re: Leaf: machine intelligence framework in Rust
#12Great work -- very exciting to see a Rust ML framework! Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is) but TensorFlow (like Theano) is graph/flow-based rather than layer-based. I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes. Do you worry that by adopting a layer-ba…
I am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky.
I think there is still a lot of room for improvement for layer-based networks, but even if the graph-based approach will become more prelevant I think the Rust trait system gives us great tools to migrate and reuse a lot from our layer based approach.
Re: Leaf: machine intelligence framework in Rust
#13Since there seems to be an uptick in submissions related to machine intelligence projects lately (or rather since Google released http://tensorflow.org ) (e.g. https://news.ycombinator.com/item?id=10537697 ) does anyone dare to make a comparison? I am currently just getting started in this area due to my interest in AI and I am not quite sure which route to choose.
I highly recommend Weka, http://www.cs.waikato.ac.nz/~ml/weka/. I used it for river flow forecasting (published a paper) and it was very good. We ran about 100 different models, various neural nets, SVM, logistic regression, multilinear regression, etc.
I spent about a month trying different tools and liked the understandability of Weka. Scikit-learn calls out to C for the heavy lifting, which is great but it's not exactly easy to read and understand the code.
Re: Leaf: machine intelligence framework in Rust
#14Exciting to see this! I haven't taken a look at TensorFlow code yet but it looks very similar (method names and all) to Caffe. I've wanted a CNN library for Rust for a while. It sounds like performance is important to you. Are there any plans to add GPU paths soon? If it's CUDA, are you going to use cuDNN?
Yes, GPU support has high priority but we are still investigating what library/tools we want to go with, since the current options in Rust are not that great. Maybe we'll use Arrayfire, maybe we'll use CUDA/cuDNN (would require writing Rust binding) or maybe something else.
Re: Leaf: machine intelligence framework in Rust
#15I am very excited to finally see this type of library in a modern (strongly typed) language, let alone my current passion language. In my opinion your pitch (ok, Readme), should put emphasis on the productivity and usability gain of types and a generally "smarter" language moreso than the not-offloaded-to-C performance (do people care what they code gets compiled to?) I was pretty excited about the tensorflow announc…
Thanks for the feedback!
We love Rust for a lot of reasons, but couldn't really fit them all in the Readme ;)
Re: Leaf: machine intelligence framework in Rust
#16Exciting to see this! I haven't taken a look at TensorFlow code yet but it looks very similar (method names and all) to Caffe. I've wanted a CNN library for Rust for a while. It sounds like performance is important to you. Are there any plans to add GPU paths soon? If it's CUDA, are you going to use cuDNN?
Yes, GPU support has high priority but we are still investigating what library/tools we want to go with, since the current options in Rust are not that great. Maybe we'll use Arrayfire, maybe we'll use CUDA/cuDNN (would require writing Rust binding) or maybe something else.
If you're going for GPU support then please do not use CUDA. It's non-portable (NVIDIA only), proprietary, clunky and it requires a huge SDK which is very awkward to install. I highly recommend OpenCL which is portable, requires no SDK, is open and for properly optimized code is just as fast as CUDA.
Re: Leaf: machine intelligence framework in Rust
#17Earlier quoted context omitted.
Yes, GPU support has high priority but we are still investigating what library/tools we want to go with, since the current options in Rust are not that great. Maybe we'll use Arrayfire, maybe we'll use CUDA/cuDNN (would require writing Rust binding) or maybe something else.
If you're going for GPU support then please do not use CUDA. It's non-portable (NVIDIA only), proprietary, clunky and it requires a huge SDK which is very awkward to install. I highly recommend OpenCL which is portable, requires no SDK, is open and for properly optimized code is just as fast as CUDA.
Another nice thing about OpenCL is that it is also supported by non-GPU hardware. Kronos maintains a pretty impressive list: https://www.khronos.org/conformance/adopters/conformant-prod...
Re: Leaf: machine intelligence framework in Rust
#18Great work -- very exciting to see a Rust ML framework! Dist-Belief (TensorFlow's predecessor) was oriented around layers (as Leaf is) but TensorFlow (like Theano) is graph/flow-based rather than layer-based. I think in particular the layer-based approach makes it awkward to represent more complicated models since not every model can be decomposed into backward/forward passes. Do you worry that by adopting a layer-ba…
I am very optimistic concerning our plans to make Leaf work in a distributed fashion. It's not quite finalized yet, but the rough outline consists of abstracting other parts of the Network away as a special type of layer and using capnproto + zeromq for transportation. Sadly I haven't come across a good implementation of model parallelism to learn from, so that might become a bit tricky. I think there is still a lot…
I just curious about is there any good Actor model library in Rust, just like Akka for Scala/Java, so there is a better model for transportation.
Re: Leaf: machine intelligence framework in Rust
#19I like your idea very much, especially implement in Rust. As we need more powerful computation, the native low-level language is a very good choose, we can control the memory, and call the native library to run on CPU/GPU.
Even the development of Rust may not very high enough compared with Java/Go(I do not sure), but we can keep the core as simple and steady, and use high-level language such as Python/R to implement the API.
Hope to hear more wonderful news about this Project.
Re: Leaf: machine intelligence framework in Rust
#20Earlier quoted context omitted.
Yes, GPU support has high priority but we are still investigating what library/tools we want to go with, since the current options in Rust are not that great. Maybe we'll use Arrayfire, maybe we'll use CUDA/cuDNN (would require writing Rust binding) or maybe something else.
If you're going for GPU support then please do not use CUDA. It's non-portable (NVIDIA only), proprietary, clunky and it requires a huge SDK which is very awkward to install. I highly recommend OpenCL which is portable, requires no SDK, is open and for properly optimized code is just as fast as CUDA.
While this sounds attractive, OpenCL performance just doesn't seem to be there at the moment.
In CNN benchmarks[1] the CL based implementations consistently finish at the bottom. I think this is mostly because of the weak nVidia OpenCL implementations, but that doesn't help, since most people use nVidia (eg, Amazon GPU instances are nVidia based)