Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is deep learning a new kind of programming?

tomasp.net

Post reply on HN
Print view View original

61–66 of 66 posts

Re: Is deep learning a new kind of programming?

#61

Post by xapata »

_nhynes wrote: post #41
xapata wrote: post #29

Earlier quoted context omitted.

What other statistical machine learning algorithm gives you a state machine as an output? The particular encoding -- matrix or otherwise -- is just an implementation detail.

> What other statistical machine learning algorithm gives you a state machine as an output? * hidden Markov models * autoregressive models * learned LQR Sure, some of those have finite memory, but RNNs are practically limited–not as easily quantified. I see what you're saying about the single-step operation appearing unique, but I also think that RNNs can be viewed through a lens such that they look like "normal" pro…

I'm not familiar with LQRs. Versus HMMs and linear autoregressive models, I think avoiding the need to predefine the form of the autoregression is a bigger leap than it might appear right now. I'd always been a neural network skeptic, thinking they're excessively complex for no benefit. It seems to me that's no longer true. I'm not certain that'll hold, but it's exciting.

Re: Is deep learning a new kind of programming?

#62

Post by dr_zoidberg »

CuriouslyC wrote: post #55
teruakohatu wrote: post #52

Earlier quoted context omitted.

> There's no rule that when the number of parameters is small deep learning shouldn't be used I would be genuinely interested in examples of problems with a very low number of predictors (say two to five) when a neutral net would be appropriate (where as you say less complex methods have been tried and failed). I just can't think of one.

Historically the XOR function has been the simple example that many ML algorithms can't handle. Just imagine a higher dimensional XOR with outliers, and you have a pretty good use case for DL with limited predictors.

Historically, this was solved in the 80's with the multi-layer perceptron, but it seems it still gets repeated.

Re: Is deep learning a new kind of programming?

#63

Post by HexagonalKitten »

programmerslave wrote: post #26

Why can’t there be a discussion on machine learning without everyone on HN trying to prove how unnecessary it is. Queue the anecdotes on simpler regression based methods, over paid scientists, and how much superior some other simpler method is.

I've seen the same in discussions of many things from programming to cars.

For instance, FP vs Imperative, where a simple example (map for instance) is being shown and someone always compares it to a for loop, and then disses garbage collection. They're obviously thinking of their existing language (we know from the GC comment) and not of whatever point the OP is trying to make about composability or whatever.

To some degree, examples are to blame. They often aren't good and don't show, to the actual audience, what the author intended. In my FP example, you need something that can't get stuck in a syntax-level debate and is complex enough for the benefits to show through. Like doing something that would take a 50-level deep stack of for loops in an unnested fashion.

It gives you more respect for great teachers who can pull examples out of the air that both illustrate and refuse to misdirect.

Can you (because I certainly can't) write a good post or blog about the minimum useful deep learning project that can't be done equivalently via any other methods?

Re: Is deep learning a new kind of programming?

#64

Post by Barrin92 »

jpcooper wrote: post #33
Barrin92 wrote: post #15

I wouldn't say that deep learning is programming. I think the key feature of programming is legibility . A program is something that is clear enough to read and understand, to be decomposed in its constituent parts, and that has understandable semantics. For example, writing an algorithm that has precise steps and procedures is programming. Putting my input into a box, shaking the box, and taking the result out is no…

Going by the operationalistic idea of the article, which I interpret as duck typing, I think the key operations that can be applied to programmes are building them, running them, and measuring the build and results of the programme with respect to a certain goal. This applies both to a traditional programme, and to deep learning. Programmes are after all written to satisfy human needs, even if it is an obfuscated C c…

>We are perfectly happy to give money to people for them to do things without fully understanding their inner thoughts and the processes behind those thoughts.

yes, but we wouldn't say that we're programming them, which is the problem with the operational definition, it applies to everything. If my drunk uncle is great at horse-betting and I just need to give him a nice sixpack of microbrew and get measurable net worth increase out I've not turned into a computer scientist.

Hence my argument that legibility is what matters. Programmers must be able to reason, and rearrange, and understand relationship between syntax and semantics of a program.

I think it's more accurate to compare deep learning to running a sort of physical experiment, rather than programming.

Re: Is deep learning a new kind of programming?

#65

Post by jpcooper »

Barrin92 wrote: post #64
jpcooper wrote: post #33

Earlier quoted context omitted.

Going by the operationalistic idea of the article, which I interpret as duck typing, I think the key operations that can be applied to programmes are building them, running them, and measuring the build and results of the programme with respect to a certain goal. This applies both to a traditional programme, and to deep learning. Programmes are after all written to satisfy human needs, even if it is an obfuscated C c…

>We are perfectly happy to give money to people for them to do things without fully understanding their inner thoughts and the processes behind those thoughts. yes, but we wouldn't say that we're programming them, which is the problem with the operational definition, it applies to everything. If my drunk uncle is great at horse-betting and I just need to give him a nice sixpack of microbrew and get measurable net wor…

I agree that deep learning is not programming, and that maybe the title of the post is wrong. However, I agree with the sentiment of the article. The contexts that both operations are carried out in are closer together than the context of employing your uncle is to either. I was trying to highlight that it is more useful to focus on the similarity of the contexts and the net benefit of either method.

Also, if your uncle is better than your computer, then stop programming it at all. However, if he was actually any good, then he shouldn't be talking to you, and you shouldn't be giving him any beer. Unless he was banned by the bookmaker.

Re: Is deep learning a new kind of programming?

#66

Post by f6v »

mr_toad wrote: post #44
f6v wrote: post #25

Earlier quoted context omitted.

Deep learning is closer to linear regression than it’s to human capabilities.

> Deep learning is closer to linear regression than it’s to human capabilities. Quantitatively perhaps, in that deep learning is equivalent to many neurons, whereas a least squares model is only equivalent to a single neuron. Other than that there is not much evidence that a human brain is qualitatively different.

Deep learning is much more than a "long" network nowadays. Most of DL success comes from clever architectures, like Inception Net. AFAIK there's no evidence human brain is anything like those fancy deep NNs.
Post reply on HN