Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-technical explanation of deep learning

parand.com

Post reply on HN
Print view View original

31–40 of 139 posts

Re: A non-technical explanation of deep learning

#31

Post by zvmaz »

I have met people who think they understand a particular topic I am versed in, but actually don't. Similarly, I am often wary that I get superficial knowledge about a topic I don't know much about through "laymen" resources, and I doubt one can have an appropriate level of understanding mainly through analogies and metaphors. It's a kind of "epistemic anxiety". Of course, there are "laymen" books I stumbled upon which I think go to appropriate levels of depth and do not "dumb down" to shallow levels the topics, yet remain accessible, like Gödel's Proof, by Ernest Nagel. I'd be glad to read about similar books on all topics, including the one discussed in this thread.

Knowledge is hard to attain...

Re: A non-technical explanation of deep learning

#32

Post by SnooSux »

jstx1 wrote: post #23

Does stuff like this help anyone? I still haven’t forgiven CGP Grey for changing the title to his 2017 ML video to “How AIs, like ChatGPT, learn”. The video is about genetic algorithms and has nothing to do with ChatGPT. (or with anything else in modern AI)

I've barely forgiven him for explaining genetic algorithms and acting like they have any relevance to contemporary ML research.

The footnote video was an alright explanation of backprop. If that were part of the main video that would have been reasonable.

I really like his history/geography videos but anything technical leave a lot to be desired. And don't get me started on Humans Need Not Apply.

Re: A non-technical explanation of deep learning

#33

Post by clarle »

Totally aware that this isn't a fully formal definition of deep learning, but one interesting takeaway for me is realizing that in a way, corporations with their formal and informal reporting structures are structured in a way similar to neural networks too.

It seems like these sort of structures just regularly arise to help regulate the flow of information through a system.

Re: A non-technical explanation of deep learning

#34

Post by uoaei »

amelius wrote: post #8

The problem with deep learning is opposite. You can understand most of it with just high school math. Advanced math is mostly useless because of the dimensionality of neural nets.

That's like saying you understand state-of-the-art CFD code because you can read Fortran.

There are many aspects to learning systems that we still don't have any kind of grasp on, and will take more than a little advanced math (statistics/probability theory, transport theory, topology, etc.) to understand as a community.

Dunning-Kruger is probably more common in spaces like this one, where people carry social capital for being able to "spin up quickly". But the true meta-skill of upskilling is turning unknown unknowns (UU) into known unknowns (KU), and then into known knowns (KK). It's not enough to just jump from UU to KK through osmosis by reading blog posts on a news aggregator, because there will still be a huge space of unknowns not covered by that approach.

Re: A non-technical explanation of deep learning

#35

Post by uoaei »

mach1ne wrote: post #18
amelius wrote: post #8

The problem with deep learning is opposite. You can understand most of it with just high school math. Advanced math is mostly useless because of the dimensionality of neural nets.

Yes, it’s really rather like alchemy in some sense. Stuff works, and often nobody knows exactly why.

"I don't follow the latest ML scaling and theory research" does not in any way equate to "these things are unknowable".

Re: A non-technical explanation of deep learning

#36

Post by jstx1 »

SnooSux wrote: post #32
jstx1 wrote: post #23

Does stuff like this help anyone? I still haven’t forgiven CGP Grey for changing the title to his 2017 ML video to “How AIs, like ChatGPT, learn”. The video is about genetic algorithms and has nothing to do with ChatGPT. (or with anything else in modern AI)

I've barely forgiven him for explaining genetic algorithms and acting like they have any relevance to contemporary ML research. The footnote video was an alright explanation of backprop. If that were part of the main video that would have been reasonable. I really like his history/geography videos but anything technical leave a lot to be desired. And don't get me started on Humans Need Not Apply.

> And don't get me started on Humans Need Not Apply.

Well now you have to tell us. :) Many of the concrete examples in that video are exaggerated and/or misunderstood but the general question it asks - what to do when automation makes many people unemployable through no fault of their own - seems valid.

Re: A non-technical explanation of deep learning

#37

Post by jedberg »

fifteen1506 wrote: post #27
giardini wrote: post #5

Nothing about LLMs?!

Yeah, I need something to explain me about those Transformers things. I know it was published by Google in 2017 and that it is 'magic'. End of knowledge. Maybe I should ask ChatGPT?

> Maybe I should ask ChatGPT?

You actually should, it spits out a pretty good explanation (sometimes).

Re: A non-technical explanation of deep learning

#38

Post by sainez »

zvmaz wrote: post #31

I have met people who think they understand a particular topic I am versed in, but actually don't. Similarly, I am often wary that I get superficial knowledge about a topic I don't know much about through "laymen" resources, and I doubt one can have an appropriate level of understanding mainly through analogies and metaphors. It's a kind of "epistemic anxiety". Of course, there are "laymen" books I stumbled upon whic…

I find the best way to learn technical topics is to build a simplified version of the thing. The trick is to understand the relationship between the high level components without getting lost in the details. This high level understanding then helps inform you when you drill down into specifics.

I think this book is a shining example of that philosophy: https://www.buildyourownlisp.com/. In the book, you implement an extremely bare-bones version of lisp, but it has been invaluable in my career. I found I was able to understand nuanced language features much more quickly because I have a clear model of how programming languages are decomposed into their components.

Re: A non-technical explanation of deep learning

#39

Post by sainez »

If anyone is looking for a quick overview of how LLMs are built, I highly recommend this video by Steve Seitz: https://www.youtube.com/watch?v=lnA9DMvHtfI.

It does an excellent job of taking you from 0 to a decent understanding without dumbing down the content or abusing analogies.

Re: A non-technical explanation of deep learning

#40

Post by 0xBABAD00C »

clarle wrote: post #33

Totally aware that this isn't a fully formal definition of deep learning, but one interesting takeaway for me is realizing that in a way, corporations with their formal and informal reporting structures are structured in a way similar to neural networks too. It seems like these sort of structures just regularly arise to help regulate the flow of information through a system.

There is research claiming the entire universe is a neural network: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7712105/
Post reply on HN