Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is deep learning a new kind of programming?

tomasp.net

Post reply on HN
Print view View original

41–50 of 66 posts

Re: Is deep learning a new kind of programming?

#41

Post by _nhynes »

xapata wrote: post #29
_nhynes wrote: post #18

Earlier quoted context omitted.

RNNs provided by common frameworks really do give a matrix back (examples [0, 1]). They're about as stateful as any other object or generator function. [0]: https://pytorch.org/docs/stable/generated/torch.nn.RNN.html [1]: https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cu...

What other statistical machine learning algorithm gives you a state machine as an output? The particular encoding -- matrix or otherwise -- is just an implementation detail.

> What other statistical machine learning algorithm gives you a state machine as an output?

* hidden Markov models

* autoregressive models

* learned LQR

Sure, some of those have finite memory, but RNNs are practically limited–not as easily quantified.

I see what you're saying about the single-step operation appearing unique, but I also think that RNNs can be viewed through a lens such that they look like "normal" programming concepts like generators and folding iterators.

Re: Is deep learning a new kind of programming?

#42

Post by ramraj07 »

chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

Was there any rationale to using a "deep learning model" to train a 20 parameter model? Sounds like some amateur DS convinced the team this is a good idea because he thought deep learning was cool?

Re: Is deep learning a new kind of programming?

#44

Post by mr_toad »

f6v wrote: post #25
ipsum2 wrote: post #5

Earlier quoted context omitted.

How well does "linear least squares to some nonlinear methods, simulated annealing, bayesian methods" work when you're doing speech to text, text generation or object detection? Deep learning is different in that it's a huge leap closer to human capabilities compared to the other methods you've listed.

Deep learning is closer to linear regression than it’s to human capabilities.

> Deep learning is closer to linear regression than it’s to human capabilities.

Quantitatively perhaps, in that deep learning is equivalent to many neurons, whereas a least squares model is only equivalent to a single neuron. Other than that there is not much evidence that a human brain is qualitatively different.

Re: Is deep learning a new kind of programming?

#45

Post by tluyben2 »

teruakohatu wrote: post #11
chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

That kind of problem, with such a limited number of parameters, really shouldn't be thrown into a neutral network. A decision tree (or varient) might have been the ideal ML technique, and you may have been able quickly see what parameters mattered and reduce the four parameters to code if needed. Neural networks make sense with huge number of input parameters where feature selection is really tricky to reason about a…

When I studied AI in uni in the cold cold (cold) winter of AI and this kind of input was really significant, but most problems we consider ML now are vastly more complex and other problems can be addressed by things that are no longer considered AI at all (while they were back then).

It is funny how my uni top research (on 1m$ computers) neural nets are considered to make no sense anymore. That went a lot faster than programming.

Re: Is deep learning a new kind of programming?

#46

Post by adrian_b »

programmerslave wrote: post #38
fxtentacle wrote: post #31

Earlier quoted context omitted.

It's an expensive (hardware, time, complexity) technique that is rarely the best. Why wouldn't people discuss cheaper, faster, understandable alternatives?

Just because you aren’t employed in a job that can make use of deep learning doesn’t mean it isn’t profitable. It’s extremely profitable. I’m tired of reading about some dumb alternatives that aren’t even relevant to the topic. Deep learning works very well for a certain class of problems. Continuing on with the trope that it doesn’t work just shows you aren’t educated

While I agree with you that deep learning works very well for a certain class of problems and that some criticism directed towards it is misguided, there are also some arguments for the opposite side.

I normally do not feel the need to comment about deep learning, because it is only tangentially related to some of my past projects, but I can also understand those who might want to comment negatively, because I have seen many cases of managers who did not understand at all how exactly certain problems can be solved, but nevertheless they pushed vigorously for the use of deep learning to replace other better suited solutions, because they believed it to be a modern and universally better method.

So there were times when I was tired of seeing one more attempt to misuse deep learning and to have to explain and demonstrate once more which solution is better.

Obviously, any such opinions, about which method is better in a certain case, should be proved with numerical results from tests or simulations, not with guesses, but some times that requires a lot of work to implement both methods, even if you are pretty sure about which will be the result.

Re: Is deep learning a new kind of programming?

#47

Post by mr_toad »

teruakohatu wrote: post #2

Is linear regression programming? A hammer, nail, saw and timber can also be used to solve problems but I wouldn't call those programming in and of themselves, but they could be used to built analog computers (where cogs, cams etc. Are like lines of code or procedures). Building a neutral network to get a result is not at all like programming. There is usually not a "perfect" structure, rather there are hardware, ene…

> Is linear regression programming?

I’d argue not. Mostly. If you fit a model using lm() in R, and then apply that model it’s not the same as hand selecting the weights of a linear equation and coding that equation.

You could in theory select the same weights, and code it by hand. But no one ever does.

Re: Is deep learning a new kind of programming?

#48

Post by shbm »

fnord77 wrote: post #13
chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

wouldn't principal component analysis have done the same thing without the brute forcing?

Yes. I think so. Or maybe some correlation plots. I doubt that a proper EDA was performed pre-modelling.

Re: Is deep learning a new kind of programming?

#49

Post by jlouis »

programmerslave wrote: post #26

Why can’t there be a discussion on machine learning without everyone on HN trying to prove how unnecessary it is. Queue the anecdotes on simpler regression based methods, over paid scientists, and how much superior some other simpler method is.

Most people don't really have the data set to exploit larger ML models.

And the mistake people make is that they don't start with a simple model. They move straight into the heavy ones.

Re: Is deep learning a new kind of programming?

#50

Post by suref »

teruakohatu wrote: post #11
chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

That kind of problem, with such a limited number of parameters, really shouldn't be thrown into a neutral network. A decision tree (or varient) might have been the ideal ML technique, and you may have been able quickly see what parameters mattered and reduce the four parameters to code if needed. Neural networks make sense with huge number of input parameters where feature selection is really tricky to reason about a…

There's no rule that when the number of parameters is small deep learning shouldn't be used. The one time where deep learning maybe shouldn't be attempted at all is when the number of samples is very limited. While it excels with high dimensional hierarchical data it can do well on other problems as well. It differentiates from problem to problem and usually multiple solutions are tried and compared, starting with EDA and linear regression.
Post reply on HN