Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A non-technical explanation of deep learning

parand.com

Post reply on HN
Print view View original

121–130 of 139 posts

Re: A non-technical explanation of deep learning

#121

Post by mtlmtlmtlmtl »

0xBABAD00C wrote: post #40
clarle wrote: post #33

Totally aware that this isn't a fully formal definition of deep learning, but one interesting takeaway for me is realizing that in a way, corporations with their formal and informal reporting structures are structured in a way similar to neural networks too. It seems like these sort of structures just regularly arise to help regulate the flow of information through a system.

There is research claiming the entire universe is a neural network: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7712105/

You can find theoretical physics research claiming a lot of bonkers things that almost certainly are not true, that's sort of the nature of the field.

Re: A non-technical explanation of deep learning

#122

Post by pringk02 »

tasty_freeze wrote: post #28
pringk02 wrote: post #12

Earlier quoted context omitted.

Doesn't that mean it supports the Chinese room argument? I'm not sure I follow your reasoning. (also, popular conciousness forgets that technically the Chinese Room argument is only arguing against the much narrower, and now philosophically unfashionable, "Hard AI" stance as it was held in the 70s)

> the Chinese Room argument is only arguing against the much narrower, and now philosophically unfashionable, "Hard AI" stance as it was held in the 70s Searle has stood behind his argument in the 70s, but in every decade since then too. The main failure is that most people fundamentally don't believe they are mechanistic. If one believe in dualism, then it easy to attribute various mental states to that dualism, and…

Searle's argument in the Chinese Room is only that passing the Turing Test isn't enough to prove evidence of Mind (capital 'm' to distinguish it at the philosophical jargon term, and all it entails). He does hold the stance that he doesn't think Computationalism (in the style of Dennet) is correct. I'm not sure if he personally feels the Chinese Room argument refutes that stance as a whole, but I believe the general consensus is that, as originally formulated in his essay, it does not stand as a total refutation of Computationalism, without reading between the lines or squinting your eyes a bit. Searle does have a wider stance that he does not think computations can have things equivalent to mental states, especially intentionality. Obviously there is a whole separate debate to his correctness there, but I'm skipping over it to just discuss the Chinese Room

That passing the Turing Test is not enough to exhibit evidence of Mind is not that controversial today. GPT-4 could easily pass the Turing Test as it was originally formulated. There are not many out there that think it possesses conciousness or intentionality or any mental states at all really. We'd generally agree now that passing the Turing Test is only a step towards creating an actual artificial mind (how large or small a step is still up for debate).

Anyway, all this is a tangent as I still don't understand why the original commenter feels this article provides a refutation of the Chinese Room argument when it seems (to me) to reinforce it. I'm just curious on that perspective and was interested in hearing more.

Re: A non-technical explanation of deep learning

#123

Post by Hithredin »

redog wrote: post #42
onikolas7 wrote: post #7

Funny. In the game black&white you would slap or pet your avatar to train it. The lead AI programmer on that was Demis Hassabis of deepmind fame.

Somehow he knew AI would be our Gods.

Actually, the god is the AI trainer. The AI is the god tool and us poor villagers are forced to worship the first.

Re: A non-technical explanation of deep learning

#124

Post by amelius »

ftxbro wrote: post #63
amelius wrote: post #8

The problem with deep learning is opposite. You can understand most of it with just high school math. Advanced math is mostly useless because of the dimensionality of neural nets.

> Advanced math is mostly useless because of the dimensionality of neural nets. It depends what you mean by advanced math. There is a lot of math that only really comes into play because of the high dimensionality! For example math related to tensor wrangling, low rank approximations, spectral theory, harmonic theory, matrix calculus derivatives, universality principles, and other concepts that could be interesting o…

Yes but do you have examples of "higher" math not being just a curiosity and actually making it into real world models and training algorithms?

Re: A non-technical explanation of deep learning

#125

Post by Chocobean »

sainez wrote: post #38
zvmaz wrote: post #31

I have met people who think they understand a particular topic I am versed in, but actually don't. Similarly, I am often wary that I get superficial knowledge about a topic I don't know much about through "laymen" resources, and I doubt one can have an appropriate level of understanding mainly through analogies and metaphors. It's a kind of "epistemic anxiety". Of course, there are "laymen" books I stumbled upon whic…

I find the best way to learn technical topics is to build a simplified version of the thing. The trick is to understand the relationship between the high level components without getting lost in the details. This high level understanding then helps inform you when you drill down into specifics. I think this book is a shining example of that philosophy: https://www.buildyourownlisp.com/ . In the book, you implement an…

Thanks for the link. Could you give an example of something you learned better/easier after having implemented a simplified version?

Side question: Is there a entry level build your own language model or gan type learning tool out there as well?

Re: A non-technical explanation of deep learning

#126

Post by mensetmanusman »

joe_the_user wrote: post #44
clarle wrote: post #33

Totally aware that this isn't a fully formal definition of deep learning, but one interesting takeaway for me is realizing that in a way, corporations with their formal and informal reporting structures are structured in a way similar to neural networks too. It seems like these sort of structures just regularly arise to help regulate the flow of information through a system.

Uh, The similarity of corporations and neural nets is pretty much only that both are information processing systems. An operating system or missile guidance system is far more like a corporation than a neural network. Neural networks have no memory and generally don't seek particular goals, they simply recognize, predict and generate similar instances.

Neural networks do have a memory of sorts, that’s why they improve with node size. A mathematician proved something along these lines recently.

The memory isn’t digital bits like we think of now though, but abstractions in higher dimensions.

Re: A non-technical explanation of deep learning

#127

Post by mensetmanusman »

wrs wrote: post #80
cscurmudgeon wrote: post #56

Earlier quoted context omitted.

Thats not the Chinese Room argument. The argument says just because a system processes X doesn't imply it has consciousness of X.

The flaw is the unsupported assertion that the whole system being conscious of X depends on a part of the system being conscious of X. The same assertion would fail here in the same way.

At these levels of discussion everything is asserted as axioms to see what the consequences are.

If a human isn’t conscious of Chinese, but the arrangement of paper rules is, one would have to assert that paper + human is conscious.

Re: A non-technical explanation of deep learning

#128

Post by nailer »

andysinclair wrote: post #20
nailer wrote: post #4

Does this article imply there are circumstances where a spreadsheet is a cat? What a poor example of technical writing.

He’s saying that the spreadsheet represents the “picture” of the cat in terms of pixels and RGB values etc. The algorithm/workers are not really “looking” at a picture of a cat, they are analysing and looking for patterns in the data that defines the picture of the cat.

Yes I know. The spreadsheet analogy does not work.

Re: A non-technical explanation of deep learning

#129

Post by nailer »

adrianmonk wrote: post #24
nailer wrote: post #14

Earlier quoted context omitted.

Most non technical people would think there are zero circumstances where a spreadsheet could be a cat.

It's obvious from context that it's the content of the media. To me at least. If I play you a song on Spotify and say, "Is this a saxophone?", you wouldn't say, "No, it's a iPhone running Spotify." If a policeman holds up a photograph of a person and says, "Is this the person who attacked you?", the victim doesn't say, "No, it's an 8 by 10 glossy print."

Yes but you had to think about it, as did I, and there was a moment where you went "hangon, did this person imply a spreadsheet could ever be a cat?" and thought you were 95% sure they did this very bizarre thing, it was distracting.
Post reply on HN