Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is deep learning a new kind of programming?

tomasp.net

Post reply on HN
Print view View original

11–20 of 66 posts

Re: Is deep learning a new kind of programming?

#11

Post by teruakohatu »

chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

That kind of problem, with such a limited number of parameters, really shouldn't be thrown into a neutral network. A decision tree (or varient) might have been the ideal ML technique, and you may have been able quickly see what parameters mattered and reduce the four parameters to code if needed.

Neural networks make sense with huge number of input parameters where feature selection is really tricky to reason about and decision boundaries are very non-linear such as image classification.

Edited: slight clarification

Re: Is deep learning a new kind of programming?

#12

Post by m_mueller »

chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

For up to 100ish parameters, even mixed with floating point I recommend trying the midaco solver a friend of mine develops. MINLP, ant colony method (i.e. gradient descent with many restarts). From my experience this runs circles around NNs for this class of problems (parameter optimization with relatively low complexity and/or limited amount of training data available).

Re: Is deep learning a new kind of programming?

#13

Post by fnord77 »

chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

wouldn't principal component analysis have done the same thing without the brute forcing?

Re: Is deep learning a new kind of programming?

#14

Post by [deleted] »

chriszhang wrote: post #7

We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…

[deleted]

Re: Is deep learning a new kind of programming?

#15

Post by Barrin92 »

I wouldn't say that deep learning is programming. I think the key feature of programming is legibility. A program is something that is clear enough to read and understand, to be decomposed in its constituent parts, and that has understandable semantics.

For example, writing an algorithm that has precise steps and procedures is programming. Putting my input into a box, shaking the box, and taking the result out is not programming, even if the box somehow solved the problem. Merely describing a problem and then having it solved is not enough to delineate programming, because that actually does apply to almost anything.

Re: Is deep learning a new kind of programming?

#16

Post by jokoon »

As I've understood it, normal programming is transforming an input with a program to get an output.

Machine learning is giving the input and the output to get a program.

Problem is, it's too difficult to summarize or understand the resulting program, while the program you get is tied to the output data which is never really accurate.

I'm still curious how ML specialists are approaching the task of analyzing a resulting deep neural network, and squeeze some science from it (meaning putting words on things they understand and are able to explain).

I've also read that google was using ML to test different learning models, to easily find the best model to use for a given problem. I'm not sure but it sounded like they were feeding the training model and the data into another learning model. I can't remember the details or the article or the reddit comment but it sounded quite interesting.

Re: Is deep learning a new kind of programming?

#17

Post by selestify »

jokoon wrote: post #16

As I've understood it, normal programming is transforming an input with a program to get an output. Machine learning is giving the input and the output to get a program. Problem is, it's too difficult to summarize or understand the resulting program, while the program you get is tied to the output data which is never really accurate. I'm still curious how ML specialists are approaching the task of analyzing a resulti…

You're probably talking about Google AutoML. FWIW there seems to also be Amazon SageMaker and Azure Machine Learning AutoML

Re: Is deep learning a new kind of programming?

#18

Post by _nhynes »

xapata wrote: post #6
hprotagonist wrote: post #4

I have been writing optimization solvers of many forms to solve problems in engineering for about 20 years. from "make excel do linear regression on some data" to linear least squares to some nonlinear methods, simulated annealing, bayesian methods, deep neural networks -- none of this is "a new kind of programming", it's "do a bunch of data munging, throw matrix at a function, get matrix back, interpret/plot." there…

Recurrent networks don't give you a matrix back. They return a state machine. Qualitatively different. Transitioning from a function that returns a scalar, to returning a polynomial, to returning an arbitrary function, and now to returning something stateful is a big difference. I suppose the next step is to write an algorithm that trains a Turing machine.

RNNs provided by common frameworks really do give a matrix back (examples [0, 1]). They're about as stateful as any other object or generator function.

[0]: https://pytorch.org/docs/stable/generated/torch.nn.RNN.html

[1]: https://docs.nvidia.com/deeplearning/cudnn/api/index.html#cu...

Re: Is deep learning a new kind of programming?

#19

Post by sriku »

Having thought along these lines before, I realized that i didn't get any new useful insights by throwing neutral networks and conventional programming into the same bucket. Life went on as usual in both the worlds and I stopped thinking about it.

Any insights worth learning about?

Re: Is deep learning a new kind of programming?

#20

Post by sriku »

sriku wrote: post #19

Having thought along these lines before, I realized that i didn't get any new useful insights by throwing neutral networks and conventional programming into the same bucket. Life went on as usual in both the worlds and I stopped thinking about it. Any insights worth learning about?

About the only useful genralization has been differentiable programming - I.e. AD embedded in a programming language to help mix learning some functions from data. But this is not a consequence of clubbing the aforesaid two.
Post reply on HN