Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Mathematics of Artificial Intelligence (2022)

arxiv.org

Post reply on HN
Print view View original

41–50 of 61 posts

Re: The Mathematics of Artificial Intelligence (2022)

#41

Post by martopix »

mg wrote: post #4

After a cursory glance, my feeling is that "The Mathematics of Neuronal Networks" would be a better title? In recent years, the term "Artificial Intelligence" is often used instead of "Neuronal Networks". I wouldn't be surprised if this will change again. If there is evidence that it will not and Neuronal Networks are for some reason the optimal medium for intelligence, I would love to read about it.

Because AI basically "stole" the term neural networks from neuroscience, neuroscientists now often speak of "neuronal networks" while referring to biological ones, to avoid confusion. So I would recommend to not use "neuronal networks" in this context.

Re: The Mathematics of Artificial Intelligence (2022)

#42

Post by martopix »

akomtu wrote: post #39
mellosouls wrote: post #9

Let L(uy,y) = fy denote a parametric partial differential equation with y being a parameter from a high-dimensional parameter space Y ⊆ Rp and uy the associated solution in a Hilbert space H. After a high-fidelity discretization... These AI folks are clearly very clever, but they don't actually believe that's got anything to do with how human thinking works, right?

Whenever I see "Hilbert space" remarks in a paper that doesn't concern abstract math, I roll my eyes and imagine a chef who starts his youtube video with "Today we are going to cook a salad. For better precision, we are going to use knifes of triangular shape, three dimensional plates of circular shape, and the 100% dihydrogen of oxyde solution." All this only to obfuscate the fact that the entire recipe is just chop…

Try doing quantum mechanics, even at beginners level, without a proper knowledge of Hilbert spaces.

Re: The Mathematics of Artificial Intelligence (2022)

#43

Post by martopix »

mellosouls wrote: post #9

Let L(uy,y) = fy denote a parametric partial differential equation with y being a parameter from a high-dimensional parameter space Y ⊆ Rp and uy the associated solution in a Hilbert space H. After a high-fidelity discretization... These AI folks are clearly very clever, but they don't actually believe that's got anything to do with how human thinking works, right?

It's literally called "the mathematics of ARTIFICIAL intelligence"

Re: The Mathematics of Artificial Intelligence (2022)

#44

Post by mdp2021 »

elcomet wrote: post #30
mg wrote: post #4

After a cursory glance, my feeling is that "The Mathematics of Neuronal Networks" would be a better title? In recent years, the term "Artificial Intelligence" is often used instead of "Neuronal Networks". I wouldn't be surprised if this will change again. If there is evidence that it will not and Neuronal Networks are for some reason the optimal medium for intelligence, I would love to read about it.

We usually say "neural networks" and not "neuronal networks"

I think in Greek 'neura' and 'neuron' are equivalent. The perception that '-on' indicate the single or ideal entity ("ion, electron, muon...") comes from Whewell and Faraday ('-on' ex "ienai": the "goer" particles) and fully postdates the use of 'neuron'.

Edit: also see the comment from member martopix, nearby ( https://news.ycombinator.com/item?id=30988150 )

Re: The Mathematics of Artificial Intelligence (2022)

#45

Post by mdp2021 »

akomtu wrote: post #38
mdp2021 wrote: post #15

Earlier quoted context omitted.

> Neuronal Networks are for some reason the optimal medium for intelligence No, that's not the same 'intelligence' ("General Intelligence") as the "I" side in Artificial Intelligence. The term 'intelligence' applied to Artificial Neural Networks makes sense, as such: to reach a procedural solution it takes an engineer; the engineer is said to have reached the solution because "intelligent"; ANNs are (semi-)automated…

ANNs dont have the key component of intelligence - ability to learn. If our brains worked like ANNs, they'd be wired once at the brains factory, they'd produce the same output given the same input, and learning anything new would need a complete rewiring at the factory, possibly erasing other learnings (often unpredictably). Even if ANNs could update themselves on the fly, in a reasonably incremental manner, they'd j…

Well this does not mean that some mechanisms similar to those we are studying cannot be part of the original or that they cannot be part of a General Intelligence system. Even if that were simply "adjusting a structure to function and information retention". The very "compositional" nature of the perceived that makes Convolutional ANNs work was noted after biomimicry.

This, of course, as an aside.

Re: The Mathematics of Artificial Intelligence (2022)

#46

Post by mistrial9 »

woopwoop wrote: post #36

Thanks, this is great. Poorly thought out morning spitball coming. One of the reasons approximation theorems are so unsatisfying is that they are always of the form "for function class X there exists an architecture A of complexity O(N) such that blah". And then this is compared favorably with some other function class whose dimension is O(N). But there's something tricky about this: you leave the architecture unspec…

more spitball -- the term "AI" changes the context from machine-side to application-side. "AI" implies to the listener that the machine is doing something that only thinking humans could do.. emphasis on the humans, while sets of math that seek minimized error, emphasize the behavior of the functions.

Re: The Mathematics of Artificial Intelligence (2022)

#47

Post by sdenton4 »

akomtu wrote: post #38
mdp2021 wrote: post #15

Earlier quoted context omitted.

> Neuronal Networks are for some reason the optimal medium for intelligence No, that's not the same 'intelligence' ("General Intelligence") as the "I" side in Artificial Intelligence. The term 'intelligence' applied to Artificial Neural Networks makes sense, as such: to reach a procedural solution it takes an engineer; the engineer is said to have reached the solution because "intelligent"; ANNs are (semi-)automated…

ANNs dont have the key component of intelligence - ability to learn. If our brains worked like ANNs, they'd be wired once at the brains factory, they'd produce the same output given the same input, and learning anything new would need a complete rewiring at the factory, possibly erasing other learnings (often unpredictably). Even if ANNs could update themselves on the fly, in a reasonably incremental manner, they'd j…

You're asking for online learning. This is certainly possible with modern techniques - define an unsupervised loss and update the weights with gradients as new data is encountered. Federated learning also works along these lines.

'Raw' online learning is unpopular because you have no garauntee that the network won't do something funky in the field.

That said, I think there are production systems in the world which learn on a day by day basis. Eg, take all of the logs from the last day and use them to update the production model for tomorrow. Then there's enough data that you don't risk a bad step. Think of it as learning while dreaming...

Re: The Mathematics of Artificial Intelligence (2022)

#48

Post by alok-g »

mdp2021 wrote: post #22

Does anybody know of textbooks, or articles, material etc. that focus on the /consequences/ of the mathematics of ANNs? For example: why in this problem using more layers is more or less efficient than using bigger layers, or branching evolutions of the outcomes as opposed to intensive computation of a single flow etc. A reasoned summary of the tricks, in a way ("this works because of that").

Check this out:

https://ai.facebook.com/blog/advancing-ai-theory-with-a-firs...

HN discussion:

https://news.ycombinator.com/item?id=27559017

My own review comment:

https://news.ycombinator.com/item?id=27564506

Edit: Corrected the third link. Thanks mdp2021 for notifying me of the copy-paste error.

Re: The Mathematics of Artificial Intelligence (2022)

#50

Post by tomrod »

mdp2021 wrote: post #44
elcomet wrote: post #30

Earlier quoted context omitted.

We usually say "neural networks" and not "neuronal networks"

I think in Greek 'neura' and 'neuron' are equivalent. The perception that '-on' indicate the single or ideal entity ("ion, electron, muon...") comes from Whewell and Faraday ('-on' ex "ienai": the "goer" particles) and fully postdates the use of 'neuron'. Edit: also see the comment from member martopix, nearby ( https://news.ycombinator.com/item?id=30988150 )

You have the correct etymology, but the convention is for "Neural Network" in ML and "Neuronal Network" in Neuroscience for we laymen.
Post reply on HN