I tried going through the site and also the comments but couldn't wrap my head around what this library actually is. It sounds awesome based on the response/comments. Can anyone explain this to a layman?
It looks like to me the big win here is that this is a ML library just like all the others, except that the code is not written for a specific type of processor (GPU versus CPU), nor is it written for a specific compute level (desktop, server, phone). It's target is to be a general purpose ML library. This should have the affect of getting ML solutions into useable applications more quickly, since you don't have to develop on one platform, then port to another. It just works everywhere. Best guess on my part based on what I'm reading on the site.
I'm looking at the RNN implementation right now ( https://github.com/tensorflow/tensorflow/blob/master/tensorf... ). It looks like the loop over the time frames is actually in Python itself. for time, input_ in enumerate(inputs): ... This confuses me a bit. Maybe the shape is not symbolic but must be fixed. I also haven't seen some theano.scan equivalent. Which is not needed in many cases when you know the shape in a…
I think this loop actually still only builds the graph -- what `scan` would do. The computation still happens outside of python. That is, in tensorflow they perhaps don't need `scan` because a loop with repeated assignments "just works"... Let's try this: It seems like in TensorFlow you can say: import tensorflow as tf sess = tf.InteractiveSession() # magic incantation state = init_state = tf.Variable(1) # initialise…
Why wouldn't this work in Theano?
>>> import theano
>>> import theano.tensor as T
>>> state = theano.shared(1.0)
>>> states = []
>>> for step in range(10):
>>> state = state + state
>>> states.append(state)
>>>
>>> f = theano.function([], states)
>>> f()
[array(2.0),
array(4.0),
array(8.0),
array(16.0),
array(32.0),
array(64.0),
array(128.0),
array(256.0),
array(512.0),
array(1024.0)]
Nobody forced us to open-source Lasagne, so I think that remark was a bit unfair. If we really didn't care about anything but graduating, why would we bother going through the trouble of sharing the code in the first place? But I do see your point. Google obviously has a lot more manpower to spend on this, so it might be a better bet in the long run. It's also worth comparing this to a few similar projects that have…
Actually lasagne is pretty good, I wasn't targeting you (but then you're pretty good at winning kaggle competitions so perhaps there's a connection there, no?)... I'm thinking mostly of Theano, which, from a performance standpoint, appears to have died the death of a thousand inexperienced cooks in the kitchen. The ~1000x performance regressions that it invokes when a junior data scientist goes off the rails and ends…
Pixel's Law? what s that? can' t find it on google...
There are artificial and biological neural networks. The artificial one is based upon the biological one. Therefore dropping the 'artificial' or 'biological' seems fine to me.
This is practical, however I strongly disagree with this semantically. The term should always be accompanied with 'artificial' or another name should be invented as they are vastly different fields. Search Neural Network, I assure you that you will no longer find biological information; it is littering.
I think this loop actually still only builds the graph -- what `scan` would do. The computation still happens outside of python. That is, in tensorflow they perhaps don't need `scan` because a loop with repeated assignments "just works"... Let's try this: It seems like in TensorFlow you can say: import tensorflow as tf sess = tf.InteractiveSession() # magic incantation state = init_state = tf.Variable(1) # initialise…
Why wouldn't this work in Theano? >>> import theano >>> import theano.tensor as T >>> state = theano.shared(1.0) >>> states = [] >>> for step in range(10): >>> state = state + state >>> states.append(state) >>> >>> f = theano.function([], states) >>> f() [array(2.0), array(4.0), array(8.0), array(16.0), array(32.0), array(64.0), array(128.0), array(256.0), array(512.0), array(1024.0)]
Thanks! When I tried this before, I thought compilation was stuck in an infinite loop and gave up after about a minute. But you're right, it works. Though on my machine, this took two and a half minutes to compile (ten times as long as compiling a small convnet). For 10 recurrence steps, that's weird, right? And the TensorFlow thing above runs instantly.
This is really significant. At this moment in history, the growth of computer power has made a bunch of important signal-processing and statistical tasks just feasible, so we are seeing things like self-driving cars, superhuman image recognition, and so on. But it's been very difficult to take advantage of the available computational power, because it's in the form of GPUs and clusters. TensorFlow is a library design…
Agreed, this is huge . I've been thinking along these lines for a while, but more focused on traditional programming systems. The approach of programming via data-flows and visually representing those is going to be enormous in the coming decade, IMHO. Especially in asynchronous and heterogenous programming environments. Actually, it feels like I've been "scooped" somewhat. However, at the same time it's awesome to f…
Yeah ... I'm a big fan of data-flow programming (Been working on my own approach for a little while too ...)
In my domain (finance) correlations between vectors are unstable but (maybe) dependent on cross-sectional relationships in the problem space. Some of the mathematics behind elastic body deformation (car tyres in in mechanical engineering, fluid dynamics in weather forecasting) have high applicability. Tensors are required. It's true that tensors are hard to reason about - they overclock my brain most of the time - bu…
It's not that tensors are particularly hard to reason about. It's just that the way they are represented as multidimensional arrays hides very well the fact that they are element of a tensor product (hence tensors). Natural operations like, well, the tensor product are often not even available.
I agree that tensors are often used simply as a convenient representation for multiple lower-dimensional objects. I suspect that is because there is still plenty of value in many fields, including mine, in exploring scalar, vector, and matrix representations of problems, and tensors are often unexploited. They're used as convenient data structures, not as algorithmic necessities. Still, as more and more people have access to data and explore it, increasing competition, reducing the "alpha" of simpler analyses, moving deeper into dimensionality on algorithms will be the only choice for those who want to innovate, is my view. So it is probably necessary to have the tools at our disposal already.
The programming model here is very similar to that of the language I work on, Streams Processing Language, normally just called SPL. A paper we have submitted on the language itself: http://www.scott-a-s.com/files/ibm_tr_2014.pdf
Our domain is online stream processing for, generally, the big data space. I do think, however, that describing computations in this manner gives enormous flexibility to runtime systems to actually exploit all available parallelism dynamically.