Is deep learning a new kind of programming?
1–10 of 66 posts
Re: Is deep learning a new kind of programming?
#2Is linear regression programming?
A hammer, nail, saw and timber can also be used to solve problems but I wouldn't call those programming in and of themselves, but they could be used to built analog computers (where cogs, cams etc. Are like lines of code or procedures).
Building a neutral network to get a result is not at all like programming. There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network.
Re: Is deep learning a new kind of programming?
#3[deleted]
Re: Is deep learning a new kind of programming?
#4I have been writing optimization solvers of many forms to solve problems in engineering for about 20 years. from "make excel do linear regression on some data" to linear least squares to some nonlinear methods, simulated annealing, bayesian methods, deep neural networks -- none of this is "a new kind of programming", it's "do a bunch of data munging, throw matrix at a function, get matrix back, interpret/plot."
there's really no other magic in it than that.
Re: Is deep learning a new kind of programming?
#5I have been writing optimization solvers of many forms to solve problems in engineering for about 20 years. from "make excel do linear regression on some data" to linear least squares to some nonlinear methods, simulated annealing, bayesian methods, deep neural networks -- none of this is "a new kind of programming", it's "do a bunch of data munging, throw matrix at a function, get matrix back, interpret/plot." there…
How well does "linear least squares to some nonlinear methods, simulated annealing, bayesian methods" work when you're doing speech to text, text generation or object detection? Deep learning is different in that it's a huge leap closer to human capabilities compared to the other methods you've listed.
Re: Is deep learning a new kind of programming?
#6I have been writing optimization solvers of many forms to solve problems in engineering for about 20 years. from "make excel do linear regression on some data" to linear least squares to some nonlinear methods, simulated annealing, bayesian methods, deep neural networks -- none of this is "a new kind of programming", it's "do a bunch of data munging, throw matrix at a function, get matrix back, interpret/plot." there…
Recurrent networks don't give you a matrix back. They return a state machine. Qualitatively different.
Transitioning from a function that returns a scalar, to returning a polynomial, to returning an arbitrary function, and now to returning something stateful is a big difference. I suppose the next step is to write an algorithm that trains a Turing machine.
Re: Is deep learning a new kind of programming?
#7We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results.
So he replaced the model with a single line of code with one boolean expression made with those 4 parameters connected with logical operators.
Re: Is deep learning a new kind of programming?
#8We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…
Have you tried training the network using dropout?
Re: Is deep learning a new kind of programming?
#9I would go a step further, and say that prompt design will become an important sector of programming.
Modern language models (eg GPT-3 et al) offer the capability to take a natural language input, match it against the context of the sentence, then propose a query that is understandable to the layperson. This abstraction allows us to understand the problem better, rather than just analyzing the way the problem manifests itself in code. Having a programming language that mirrors our everyday communication is an important step forward in making the innovations from software broadly available.
The next wave of programmers will need to understand how human language can be used to efficiently guide models to solve problems that can’t be solved by human-written code. This is a big challenge, and we’re just at the very beginning of it, but I think it will open up new and undiscovered ways to create value in the world.
I have quite a few additional thoughts on the topic which I’ve captured here: https://sundayscaries.substack.com/p/whos-the-real-expert
Re: Is deep learning a new kind of programming?
#10We trained a deep learning model to look at like 20 system parameters and predict an output. the parameters were binary. So one curios engineer decided to brute-force the trained model with all possible inputs like 2^20 inputs to see what the model does. he found for the problem we were solving only 4 of the 20 parameters had effect on results. the remaining approx 16 parameters do not affect results. So he replaced…
Have you tried training the network using dropout?
yes, dropout and other regularization techniques were used