Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meet the algorithm that can learn “everything about anything”

gigaom.com

Post reply on HN
Print view View original

61–70 of 81 posts

Re: Meet the algorithm that can learn “everything about anything”

#61

Post by pavpanchekha »

PurplePanda wrote: post #7
pavpanchekha wrote: post #4

Earlier quoted context omitted.

This is in fact a classic criticism of modern AI. Classic approaches tried to replicate human reasoning mechanisms. The work was very cool, but far harder to understand and scale, compared to modern statistical algorithms. To some extent, you are faced with a choice: a small palette of mediocre techniques whose workings are beautiful theories of the human mind; or, a broadly-applicable, incredibly practical set of to…

I understand that other being hard to scale in a machine efficiency sense, the old techniques were also rather limited in applicable domain. They were often based on unjustified models of whatever the author decided was a good model of thought/reasoning, whether "frames", predicate calculus, constraint propagation or whatever. It seems to me that although you say statistical methods have little motivation, the motiva…

That's actually what I meant by scale—that they were slow isn't really a problem if you're interested in AI for the raw fun of it. While justifications for using logic or constraints or probability theory were often lacking from classical techniques, do understand that logic itself was originally developed as a crystallization of proper human thought. And many of the classic AI folks did care deeply about understanding how humans achieved certain results. For example, take a look at Marvin Minsky's work. To the contrary, modern techniques aren't interested in replicating human thought; they are simply interested in replicating human results.

Re: Meet the algorithm that can learn “everything about anything”

#62

Post by baddox »

narrator wrote: post #37
baddox wrote: post #5

Earlier quoted context omitted.

I don't think there is a meaningful distinction. Do chess computers have a "fundamental understanding" of chess, which humans traditionally considered a benchmark of human intelligence/strategy? Analogous to the philosophical zombie thought experiment, I think that "real intelligence/understanding" is indistinguishable from simply being able to perform actions to accomplish the same tasks that humans traditionally co…

The fundamental problem humans will have with strong AI is that it won't be able to properly rationalize it's actions. Sure you'll have a statistical model for why it did what it did but that means about as much to anybody as a numerical reification of the quantum states of every particle of a person's mind at the time they made a decision.

When you say "rationalize its actions," are you referring to some mechanism in the inner workings of the brain/software? Or are you just talking about saying things that help humans understand why an action was chosen? I maintain that there is no distinction between the two, and it's quite conceivable that an AI could take an action then say something that helps humans understand why that action was chosen.

Re: Meet the algorithm that can learn “everything about anything”

#63

Post by pavpanchekha »

tegeek wrote: post #16
pavpanchekha wrote: post #4

Earlier quoted context omitted.

This is in fact a classic criticism of modern AI. Classic approaches tried to replicate human reasoning mechanisms. The work was very cool, but far harder to understand and scale, compared to modern statistical algorithms. To some extent, you are faced with a choice: a small palette of mediocre techniques whose workings are beautiful theories of the human mind; or, a broadly-applicable, incredibly practical set of to…

I think one of source of intelligence in humans is "imagination". IMO "Imagination" itself isn't intelligence but an input to the last "intelligent" activity. Computers can be taught to exhibit different "intelligent" activities and that is AI & Machine learning is all about. Human imagination works without any knowledge skill or learning (it is just my understanding, and i might be wrong). Producing rough and wild-w…

Whatever your opinions on what "intelligence" really is and the role of "imagination" in producing it, I can assure you that many computer scientists would like to replicate human creativity, heuristics, and problem solving techniques. Many of the early AI pioneers were as interested in understanding human thought and understanding machine thought (source: conversations with some of them). Modern techniques simply have a different aim.

Re: Meet the algorithm that can learn “everything about anything”

#64

Post by agibsonccc »

brickmort wrote: post #58
agibsonccc wrote: post #44

Earlier quoted context omitted.

Disclaimer: I have vested interest in Deep Learning having built a distributed deep learning framework[1] and building a business around it. Deep Learning is actually worth the hype though. It has 2 main merits that are interesting. 1. Auto Trend Discovery 2. Plays very well with parallelism The main problem, which I'm hoping to fix, is feasibility and ease of use. Neural nets to the untrained eye can be a black box…

DL4J is fascinating! thanks for sharing, I'm definitely going to do some playing around with this framework.

Thanks! I will be publishing a comprehensive setup of an AWS cluster later this week. Here's the rough of it now: http://deeplearning4j.org/dl4jcluster.html

Still needs a bit of work in terms of linearity, but running it on a 20 node cluster was easy. Not going to benchmark it more than that right now, but it's really starting to come to life.

Re: Meet the algorithm that can learn “everything about anything”

#65

Post by agibsonccc »

agibsonccc wrote: post #44
zwieback wrote: post #34

Earlier quoted context omitted.

Judging by the amount of "deep learning" submissions to HN bad times for AI are already here although maybe they never left. In defense of the LEVAN thing, though, I didn't see any claims that this is science at all, more like an exploratory application illustrating an algorithm.

Disclaimer: I have vested interest in Deep Learning having built a distributed deep learning framework[1] and building a business around it. Deep Learning is actually worth the hype though. It has 2 main merits that are interesting. 1. Auto Trend Discovery 2. Plays very well with parallelism The main problem, which I'm hoping to fix, is feasibility and ease of use. Neural nets to the untrained eye can be a black box…

Seem to be getting some disagreeing with me. Could you elaborate as to why? I'm not pretending to be an all knowing person who can predict the future here. I'd especially love to hear from others in the field.

Re: Meet the algorithm that can learn “everything about anything”

#66

Post by throwawaydl »

agibsonccc wrote: post #44
zwieback wrote: post #34

Earlier quoted context omitted.

Judging by the amount of "deep learning" submissions to HN bad times for AI are already here although maybe they never left. In defense of the LEVAN thing, though, I didn't see any claims that this is science at all, more like an exploratory application illustrating an algorithm.

Disclaimer: I have vested interest in Deep Learning having built a distributed deep learning framework[1] and building a business around it. Deep Learning is actually worth the hype though. It has 2 main merits that are interesting. 1. Auto Trend Discovery 2. Plays very well with parallelism The main problem, which I'm hoping to fix, is feasibility and ease of use. Neural nets to the untrained eye can be a black box…

I think it's worth clarifying a few points here - this is a fairly naive view of deep learning being put forward.

In particular -

1) what does 'automatic trend discovery' mean? We've been able to do change-point detection, linear regression, etc. for hundreds of years. If you're talking about automatically learning a feature representation, then there are other algorithms that can do this, in a much simpler way. If you're arguing that it produces better representations, then make that argument.

2) This is almost _completely_ false, and indicates a substantial lack of experience of ML beyond deep learning. Other machine learning algorithms (SVMs, LR, even some decision tree algorithms) are _much_ easier to train in parallel - this is (partially) because your objective function has certain nice properties that allow you to combine partial solutions together that are produced in parallel (convexity, separability). When you're using gradient-based methods on an incredibly ugly non-convex function from a multi-layer neural network, you're in a completely different world.

Granted, there have been techniques coming out for training in multiple address spaces, but these are _hacks_ to get around the ugly structure of the problem, not the principled approaches that exist for other algorithms.

I don't have any perspective on your deeplearning4j library, but I'm skeptical of its utility given the existence of existing well-tested deep learning libraries written/contributed to by renowned experts in the field (e.g. cuda-convnet, caffe, torch). This stuff is a) very tricky to get right, b) very tricky to debug, and c) very performance sensitive. Just a quick pass through shows zero references to CUDA/GPGPU programming, so I'd suspect performance is going to be significantly worse than the aforementioned libraries.

Re: Meet the algorithm that can learn “everything about anything”

#67

Post by dang »

agibsonccc wrote: post #44
zwieback wrote: post #34

Earlier quoted context omitted.

Judging by the amount of "deep learning" submissions to HN bad times for AI are already here although maybe they never left. In defense of the LEVAN thing, though, I didn't see any claims that this is science at all, more like an exploratory application illustrating an algorithm.

Disclaimer: I have vested interest in Deep Learning having built a distributed deep learning framework[1] and building a business around it. Deep Learning is actually worth the hype though. It has 2 main merits that are interesting. 1. Auto Trend Discovery 2. Plays very well with parallelism The main problem, which I'm hoping to fix, is feasibility and ease of use. Neural nets to the untrained eye can be a black box…

Please post that as a Show HN when you feel you're ready. (Email hn@ycombinator.com if you want clarification of what that means.)

Edit: In case that seems foreboding at all, what I mean is: this looks really cool and will probably make an awesome Show HN, whenever you think it's ready for a post of its own.

Re: Meet the algorithm that can learn “everything about anything”

#70

Post by agibsonccc »

throwawaydl wrote: post #66
agibsonccc wrote: post #44

Earlier quoted context omitted.

Disclaimer: I have vested interest in Deep Learning having built a distributed deep learning framework[1] and building a business around it. Deep Learning is actually worth the hype though. It has 2 main merits that are interesting. 1. Auto Trend Discovery 2. Plays very well with parallelism The main problem, which I'm hoping to fix, is feasibility and ease of use. Neural nets to the untrained eye can be a black box…

I think it's worth clarifying a few points here - this is a fairly naive view of deep learning being put forward. In particular - 1) what does 'automatic trend discovery' mean? We've been able to do change-point detection, linear regression, etc. for hundreds of years. If you're talking about automatically learning a feature representation, then there are other algorithms that can do this, in a much simpler way. If y…

1. In this case, we are talking about the pretrain part of neural nets. Automatic trend discovery comes down to doing feature extraction for the user. That being said, if I was inexperienced I wouldn't be teaching this stuff[1]. Am I the best machine learning practitioner out there? No. A lot of us aren't. I am all about making other people's jobs practical though.

Yes, I am talking about learning better representations. See hinton's deep autoencoder work as a prime example of this comparing PCA to RBM based methods for topic detection[2].

2. Google and people way smarter than I am seem to be doing just fine with this[3]. That being said, I didn't say that random forest (with whole companies built on this parallelism[3]) or any of the algorithms WEREN'T friendly. I would say one of the main appeals for deep learning is the scale of data with which it can benefit from.

Feel free to be skeptical all you want, if the researchers want to take the time to write a full stack distributed framework, I welcome others in to the game. The problem with the packages out there right now, (being matlab, python) are training times, and integrating in to an actual ecosystem. I'm addressing this this year at 2 different talks[5][6].

Replying to your last point, I use blas underneath for all of the matrix calculations, I will be adding GPUs later this year, and yes you're right,this stuff is hard to make. I also wouldn't be publicizing it if I wasn't already using it in production applications. Frankly right now though, I use cpu matrices right now, because I can fire this up on AWS (without the limit of GPU RAM), and it's practical for hadoop deployments. Honestly whether we like it or not, GPUs take a lot to get right. NVIDIA[7] and AMD[8] are going to make my job pretty easy though.

To end, if I was afraid of every little obstacle, why do anything in the first place? While you're hiding behind a throw away account, I'm actually trying to put this in the hands of people who don't have the time to learn every little thing about neural networks. At the end of the day, I follow the papers very closely and enjoy what I do. I also work on all sorts of different techniques for different problems combining different machine learning algorithms for different tasks (just like anyone else would). This framework is my way of getting this out to everyone else. If you have a deep learning framework, I'd love to see it, maybe I could learn a thing or 2.

[1]: http://zipfianacademy.com/

[2]: http://www.cs.toronto.edu/~fritz/absps/esann-deep-final.pdf

[3]: https://bigml.com/

[4]: http://static.googleusercontent.com/media/research.google.co...

[5]: http://hadoopsummit.org/san-jose/schedule/

[6]: http://www.oscon.com/oscon2014/public/schedule/detail/33709

[7]: http://www.jcuda.org/jcuda/jcublas/JCublas.html [8]: http://developer.amd.com/tools-and-sdks/heterogeneous-comput...

Post reply on HN