Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-technical explanation of deep learning

parand.com

Post reply on HN
Print view View original

41–50 of 139 posts

Re: A non-technical explanation of deep learning

#41

Post by mxkopy »

0xBABAD00C wrote: post #40
clarle wrote: post #33

Totally aware that this isn't a fully formal definition of deep learning, but one interesting takeaway for me is realizing that in a way, corporations with their formal and informal reporting structures are structured in a way similar to neural networks too. It seems like these sort of structures just regularly arise to help regulate the flow of information through a system.

There is research claiming the entire universe is a neural network: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7712105/

Indra's Net is, in fact, a neural network

Re: A non-technical explanation of deep learning

#43

Post by joe_the_user »

sainez wrote: post #38
zvmaz wrote: post #31

I have met people who think they understand a particular topic I am versed in, but actually don't. Similarly, I am often wary that I get superficial knowledge about a topic I don't know much about through "laymen" resources, and I doubt one can have an appropriate level of understanding mainly through analogies and metaphors. It's a kind of "epistemic anxiety". Of course, there are "laymen" books I stumbled upon whic…

I find the best way to learn technical topics is to build a simplified version of the thing. The trick is to understand the relationship between the high level components without getting lost in the details. This high level understanding then helps inform you when you drill down into specifics. I think this book is a shining example of that philosophy: https://www.buildyourownlisp.com/ . In the book, you implement an…

I find the best way to learn technical topics is to build a simplified version of the thing. The trick is to understand the relationship between the high level components without getting lost in the details. This high level understanding then helps inform you when you drill down into specifics.

I agree but that's a good guide to build a technical understanding of a complex subject, not sufficient-in-itself tool set for considering questions in that complex subject.

Especially, I'll people combining some "non-technical summary" of quantum-mechanics/Newton Gravity/genetic engineer/etc with their personal common sense are constant annoyance to me whenever such topics come here.

Re: A non-technical explanation of deep learning

#44

Post by joe_the_user »

clarle wrote: post #33

Totally aware that this isn't a fully formal definition of deep learning, but one interesting takeaway for me is realizing that in a way, corporations with their formal and informal reporting structures are structured in a way similar to neural networks too. It seems like these sort of structures just regularly arise to help regulate the flow of information through a system.

Uh,

The similarity of corporations and neural nets is pretty much only that both are information processing systems. An operating system or missile guidance system is far more like a corporation than a neural network.

Neural networks have no memory and generally don't seek particular goals, they simply recognize, predict and generate similar instances.

Re: A non-technical explanation of deep learning

#45

Post by _gmax0 »

The most concise and intuitive line of explanation I've been given goes along the lines of this:

1 - We want to model data, representative of some system, through functions.

2 - Virtually any function can be expressed by a n-th order polynomial.

3 - We wish to learn the parameters, the coefficients, of such polynomials.

4 - Neural networks allow us to brute-force test candidate values of such parameters (finding optimal candidate parameters such that error between expected and actual values of our dataset are minimized)

Whereas prior, methods (e.g. PCA) could only model linear relationships, neural networks allowed us to begin modeling non-linear ones.

Re: A non-technical explanation of deep learning

#46

Post by somenameforme »

zvmaz wrote: post #31

I have met people who think they understand a particular topic I am versed in, but actually don't. Similarly, I am often wary that I get superficial knowledge about a topic I don't know much about through "laymen" resources, and I doubt one can have an appropriate level of understanding mainly through analogies and metaphors. It's a kind of "epistemic anxiety". Of course, there are "laymen" books I stumbled upon whic…

There are a million e.g. number parsing (image to digit) neural network type programs on GitHub. Go pick one in your preferred language and break it apart, and rebuild it, looking up the concepts behind parts you don't understand. After you finish up with the above, look up 'the xor problem' to see a common practical problem (which creating a network to replicate xor illustrates, rather than is) and you'll be well on your way to a nice fundamental understanding, built from the ground up.

One of the most interesting things about this topic is that the fundamental concepts and implementations are all really simple. It's the fact that it actually works that's mind boggling. In any case, the above is not a months like affair - but like one week of dedicated work.

Re: A non-technical explanation of deep learning

#47

Post by jacksnipe »

_gmax0 wrote: post #45

The most concise and intuitive line of explanation I've been given goes along the lines of this: 1 - We want to model data, representative of some system, through functions. 2 - Virtually any function can be expressed by a n-th order polynomial. 3 - We wish to learn the parameters, the coefficients, of such polynomials. 4 - Neural networks allow us to brute-force test candidate values of such parameters (finding opti…

Except gradient descent is about as far from brute force as it gets

Re: A non-technical explanation of deep learning

#48

Post by _gmax0 »

jacksnipe wrote: post #47
_gmax0 wrote: post #45

The most concise and intuitive line of explanation I've been given goes along the lines of this: 1 - We want to model data, representative of some system, through functions. 2 - Virtually any function can be expressed by a n-th order polynomial. 3 - We wish to learn the parameters, the coefficients, of such polynomials. 4 - Neural networks allow us to brute-force test candidate values of such parameters (finding opti…

Except gradient descent is about as far from brute force as it gets

Sure, under the assumption that your parameter space is convex.
Post reply on HN