121–130 of 133 posts
Usually I'm on the hook for being open minded about programming languages in this forum, but I'm putting my foot down. I really hope at some point this entire universe gets liberated from Python at some point. Even R would be more palatable. Both these examples are awful , error prone, obfuscated, and beholden to Python's difficulties with large sums of data.
Earlier quoted context omitted.
Have you looked at ONNX? It is a neural network exchange format that, in particular, lets you deploy PyTorch models in production via Caffe2. Here is a tutorial: http://pytorch.org/docs/master/onnx.html Disclaimer: I work on Caffe2 team (not on ONNX, though)
Does the ONNX thing actually work? I'm super skeptical of these interchanges, because it seems very difficult to avoid train/test skew. Any difference in detail between the two implementations is a potential problem. I can imagine different order of operations putting out some values by 0.1%, causing 1% eventual loss of accuracy.
Implementation details of course differ between frameworks, but luckily neural networks are very robust to noise. In my experience, changes like using Winograd/Fourier for convolutions, or even running the whole thing in FP16 do not result in noticeable artifacts, and these are among the biggest differences you could have between frameworks.
Earlier quoted context omitted.
Out of curiosity, what problems are you running into? I have never had serious problems with the 'save parameters -> dump graph -> freeze graph -> load up with C API' path with feed-forward networks or various RNNs. Either from Go or from Rust. Admittedly, the documentation in this area is extremely bad and I basically had to figure out myself how to do it, though this was long before 1.0.
The issues with the high level API goes slightly deeper. It looks like some of the graph operations are not available on Android (and equivalently on the desktop) by default[1]. The motivation for this is that we have stricter requirements for computation costs and application size on different platforms. So there is an approach that allows to compile minimal set of operations required to run the graph[2][3]. However…
The relevant code is (still) in private repositories, but I have a deck with some examples using Rust:
https://www.dropbox.com/s/t8r056f6wqlktqv/embedding-tensorfl...
Usually I'm on the hook for being open minded about programming languages in this forum, but I'm putting my foot down. I really hope at some point this entire universe gets liberated from Python at some point. Even R would be more palatable. Both these examples are awful , error prone, obfuscated, and beholden to Python's difficulties with large sums of data.
I bet you love Perl
But I'd even take Scala at this point, and you could do worse than Clojure.
Python code isn't especially "ugly" as in line noise. It's ugly in the sense that it's a deluge of outdated and ineffective programming paradigms that maximize the difficulty of writing code.
import tensorflow as tf
import numpy as np
X = tf.placeholder("float")
Y = tf.placeholder("float")
pred = tf.layers.dense(X,use_bias=false)
cost = tf.losses.mean_squared_error(labels=Y,
predictions=pred)
optimizer =
tf.train.GradientDescentOptimizer(0.01).minimize(cost)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
for t in range(10000):
x = np.array(np.random.random()).reshape((1, 1, 1, 1))
y = x * 3
(_, c) = sess.run([optimizer, cost], feed_dict={X: x, Y: y})
print cEarlier quoted context omitted.
It didn't "utterly destroy" the quotation. It's pretty easy to argue the virtues of cannibalism, it's just that for modern day society they do not trump the downsides. For example, cannibalism has successfully prevented death by starvation (e.g. the Donner Party), and it reduces the need for disposing of the dead. It's an application of reduce-reuse-recycle. Some species of other animals practice it as part of their…
Small correction: not brain parasites, but specifically prions (misfolded protein that fascinatingly self-replicate). Kuru[1] (closely related to Creutzfeldt–Jakob disease and Alzheimer's) was extensively studied last century in the Fore people of Papua New Guinea (which were practicing cannibalism as a funerary rite), culminating in the awarding of the 1997 Nobel Prize[2] for the discovery of the prion, a completely…
Earlier quoted context omitted.
I bet you love Perl
No. Julia and Haskell are my go-to suggestions. But I'd even take Scala at this point, and you could do worse than Clojure. Python code isn't especially "ugly" as in line noise. It's ugly in the sense that it's a deluge of outdated and ineffective programming paradigms that maximize the difficulty of writing code.
It's early days but so far F# it ticks all my boxes and I'm finding it an extremely nice language to write in. It is just as terse as Python (due to type inference) but statically typed, and there is a great plugin Ionide for VSCode which makes for a really polished development environment.
Plan is to use Microsoft's CNTK for ML/DL stuff.
For anyone frustrated with Python's duck typing, I highly recommend you check out F#.
An appropriate quote: "If you can't intelligently argue for both sides of an issue, you don't understand the issue well enough to argue for either." There are many people for whom the declarative paradigm is a huge plus. I would say there are at least 2 major approaches in running fast neural networks: 1. Figure out the common big components and make fast versions of those. 2. Figure out the common small components a…
please argue the opposite of this before continuing
Earlier quoted context omitted.
I have tensors in later layers whose shapes depend on the training of the previous layers. Rad! Do you have any examples (or literature) that explains when this is beneficial?
Not yet! I'm not using convnets or backprop or anything so I don't think it would be beneficial that way, but you could get something similar to what I'm doing by looking at Fritzke's Growing Neural Gas[1] [1] http://papers.nips.cc/paper/893-a-growing-neural-gas-network...