71–80 of 91 posts
Every time I decide I'm going to get into Python frameworks again, and I start looking at code, and I see people making everything object-oriented, I bail Just a personal (anti-)preference I guess
Earlier quoted context omitted.
There always has to be someone rolling out the horses-for-courses pitch. No. I wanted Lua to gain traction with other people . That's the point. I would have liked the Lua sci-ecosystem to be healthy as an alternative.
Is there an equivalent to numpy in the Lua space?
Guess there's no escaping Python. I had hoped Lua(jit) might emerge as a scientific programming alternative but with Torch now throwing its hat into the Python ring I sense a monoculture in the making. Bit of a shame really because Lua is a nice language and was an interesting alternative.
Earlier quoted context omitted.
Could you elaborate on what you find lacking in TensorFlow? I regularly use TensorFlow for exactly these sorts of dynamic graphs, and it seems to work fairly well; I haven't used Chainer or DyNet extensively, so I'm curious to see what I'm missing!
When you say "exactly these sorts of dynamic graphs", what do you mean? TensorFlow has support for dynamic length RNN unrolling but that really doesn't extend well to any dynamic graph structure such as recursive tree structure creation. Since the computation graph has a different shape and size for every input they are difficult to batch and any pre-defined static graph is likely excessive, wasting computation, or i…
I personally think the approach used in TensorFlow is preferable – having a static graph enables a lot of convenient operations, such as storing a fixed graph data structure, shipping models that are independent of code, performing graph transformations. But you're right that it entails a bit more complexity, and that implementing something like recursive neural networks, while totally possible in a neat way, ends up taking a bit more effort. I think that the trade-off is worth it in the long run, and that the design of TensorFlow is very much influenced by the long-run view (at the expense of immediate simplicity...).
The ops underlying TensorFlow's `tf.while_loop` are actually quite flexible, so I imagine you can create a lot of different looping constructs with them, including ones that easily handle recursive neural networks.
Thanks for pointing out a problem that I haven't really thought about before!
Only a few months ago people saying that the deep learning library ecosystem was starting to stabilize. I never saw that as the case. The latest frontier for deep learning libraries is ensuring efficient support for dynamic computation graphs. Dynamic computation graphs arise whenever the amount of work that needs to be done is variable. This may be when we're processing text, one example being a few words while anot…
In fact, with DyNet or PyTorth, you still need to bookkeeping the graph you traversed (tape) because no one is doing forward AD. If that's the case, why not have a good library to do symbolic computation graph and build dynamic feature on top of it. (I am not saying Tensorflow is a good symbolic computation graph library to build upon just arguing that start with a define-compile-run library doesn't necessarily hinder your ability to support dynamic graphs).
Only a few months ago people saying that the deep learning library ecosystem was starting to stabilize. I never saw that as the case. The latest frontier for deep learning libraries is ensuring efficient support for dynamic computation graphs. Dynamic computation graphs arise whenever the amount of work that needs to be done is variable. This may be when we're processing text, one example being a few words while anot…
You can build both the symbolic computation graph and do the computation at the time when defining the network architecture, thus, gaining the ability to be "dynamic" and also supporting advanced features with the symbolic representation that you built on the side. In fact, with DyNet or PyTorth, you still need to bookkeeping the graph you traversed (tape) because no one is doing forward AD. If that's the case, why n…
Earlier quoted context omitted.
You can build both the symbolic computation graph and do the computation at the time when defining the network architecture, thus, gaining the ability to be "dynamic" and also supporting advanced features with the symbolic representation that you built on the side. In fact, with DyNet or PyTorth, you still need to bookkeeping the graph you traversed (tape) because no one is doing forward AD. If that's the case, why n…
the biggest hindrance to do this are language constructs that cannot be or are inconveniently expressed in the symbolic graph, such as python's if vs tf.if and for vs theano.scan, or conditioning on some python-code (not tensor operations). So to build an eagerly evaluating symbolic graph framework that is allowed to do arbitrary things would mean that you would (to an extent) reimplement the language you are working…
Once you have a reasonable symbolic computation graph library, you don't need to explicitly build a "tape" because the symbolic representation will record the order of execution and reverse AD even graph optimization (applying CSE etc) come naturally as well.
This project aside, I'm in love with that setup UI on the homepage telling you exactly how to get started given your current setup.
Earlier quoted context omitted.
Thanks for the interesting and informative comment. Do I sense just a tiny bit of regret though? Yet another Python interface. YAPI. You heard it here first. And no, Py3 is not that nice. Too much cruft by far. And lua is miles faster than Python when you're outside the tensor domain, ie while you're sourcing and wrangling your data. Arguably luajit obviates the need for C , something you can't say about Python. Disc…
A very large portion of performance problems can be mitigated with the use of cython and the new asyncio stuff. asyncio success story: https://magic.io/blog/asyncpg-1m-rows-from-postgres-to-pytho... cython: http://scikit-learn.org/stable/developers/performance.html