Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Deep Learning with Spark and TensorFlow

databricks.com

Post reply on HN
Print view View original

21–30 of 31 posts

Re: Deep Learning with Spark and TensorFlow

#21

Post by obituary_latte »

mindcrime wrote: post #19
obituary_latte wrote: post #12

Earlier quoted context omitted.

http://i.imgur.com/boZRjbB.png

There's actually a little bit more info out there for would-be "Watson builders". https://www.ibm.com/developerworks/community/blogs/InsideSys... http://www.theregister.co.uk/2011/02/21/ibm_watson_qa_system... http://learning.acm.org/webinar/lally.cfm http://www.cs.nmsu.edu/ALP/2011/03/natural-language-processi... Of course, there's still a big gap between "Download some stuff" and "Build Watson", but at least there'…

Yup - and very grateful for those.

To me, recently, the linked graphic represents pretty well what I'm faced with on a daily basis. People seem to think that because a hammer exists, it's easy to build a house.

Re: Deep Learning with Spark and TensorFlow

#23

Post by chimtim »

orm wrote: post #14
doobwa wrote: post #9

Earlier quoted context omitted.

Why not just use GNU Parallel (or something similar) instead of Spark?

I think one reason would fault tolerance. Is there a fault tolerance layer in GNU parallel? last time I checked their homepage ( a few minutes ago), there was no reference to fault tolerance. Another reason is, perhaps, scheduling.

what fault tolerance does spark give you in this scheme? It cannot look into TF progress and checkpoint all state. Using Spark with TF, seems like an overkill -- you need to manage and install two framework what should ideally be a 200 line python wrapper or small mesos framework at most.

Re: Deep Learning with Spark and TensorFlow

#24

Post by rm999 »

amelius wrote: post #13

I have a question about neural networks. Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white. Now, by accident, all the "0" training-images have an even number of black pixels, and all the "1" training-images have an odd number of black pixels. How do you know tha…

These kinds of things can happen surprisingly easily in the real-world. The most common cause is "target leaks", which happen when the thing you are trying to predict accidentally ends up in your dataset, usually indirectly through some non-obvious process. Neural networks are especially good learners, and will perform suspiciously well in these situations.

There's often no way to know exactly what a neural network is doing, but sanity checks can catch most issues. Realistically, you wouldn't expect a neural network to perform with 100% accuracy, which would be a first clue in your example.

Re: Deep Learning with Spark and TensorFlow

#25

Post by trhway »

amelius wrote: post #13

I have a question about neural networks. Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white. Now, by accident, all the "0" training-images have an even number of black pixels, and all the "1" training-images have an odd number of black pixels. How do you know tha…

http://joi.ito.com/images2/mathjoke.gif

Re: Deep Learning with Spark and TensorFlow

#26

Post by argonaut »

mindcrime wrote: post #16
amelius wrote: post #13

I have a question about neural networks. Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white. Now, by accident, all the "0" training-images have an even number of black pixels, and all the "1" training-images have an odd number of black pixels. How do you know tha…

I would say that if you're using a single layer NN, the answer is "you don't really know". And that gets to a point about how we still don't entirely understand how neural networks work, even when they do work. If you were using a deep network though, and if the current theory is correct, it would be a slightly different story. The current thinking, as I understand it, is that with deep networks, each layer learns re…

The net is just going to learn the representation "even/odd number of black pixels," if that's the easiest thing to learn.

It also goes without saying that 2k images is probably not going to be enough data to learn any meaningfully general feature representation.

Re: Deep Learning with Spark and TensorFlow

#27

Post by argonaut »

Homunculiheaded wrote: post #17
amelius wrote: post #13

I have a question about neural networks. Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white. Now, by accident, all the "0" training-images have an even number of black pixels, and all the "1" training-images have an odd number of black pixels. How do you know tha…

There's actually a case in the early history of perceptrons that brings up this exact issue: "There is a humorous story from the early days of machine learning about a network that was supposed to be trained to recognize tanks hidden in forest regions. The network was trained on a large set of photographs – some with tanks and some without tanks. After learning was complete the system appeared to work well when “show…

I'm pretty sure that story is an urban legend. Nobody can find the original source.

Re: Deep Learning with Spark and TensorFlow

#28

Post by ole_tange »

orm wrote: post #14
doobwa wrote: post #9

Earlier quoted context omitted.

Why not just use GNU Parallel (or something similar) instead of Spark?

I think one reason would fault tolerance. Is there a fault tolerance layer in GNU parallel? last time I checked their homepage ( a few minutes ago), there was no reference to fault tolerance. Another reason is, perhaps, scheduling.

Does --retries count as fault tolerance?

Re: Deep Learning with Spark and TensorFlow

#29

Post by rahimiali »

Homunculiheaded wrote: post #17
amelius wrote: post #13

I have a question about neural networks. Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white. Now, by accident, all the "0" training-images have an even number of black pixels, and all the "1" training-images have an odd number of black pixels. How do you know tha…

There's actually a case in the early history of perceptrons that brings up this exact issue: "There is a humorous story from the early days of machine learning about a network that was supposed to be trained to recognize tanks hidden in forest regions. The network was trained on a large set of photographs – some with tanks and some without tanks. After learning was complete the system appeared to work well when “show…

the tank anecdote is also famously apocryphal. here's a good analysis of the origin of that story: http://www.jefftk.com/p/detecting-tanks

Re: Deep Learning with Spark and TensorFlow

#30

Post by conjectures »

amelius wrote: post #13

I have a question about neural networks. Say, you are training a NN to recognize handwritten characters 0 and 1, and you have 1000 training images for each character (so 2000 images in total). All images are bitmaps with 0 for black and 1 for white. Now, by accident, all the "0" training-images have an even number of black pixels, and all the "1" training-images have an odd number of black pixels. How do you know tha…

Because you can simulate from the network. If you were right, this couldn't happen:

http://www.cs.toronto.edu/~hinton/adi/index.htm

Post reply on HN