Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Mathematics of Artificial Intelligence (2022)

arxiv.org

Post reply on HN
Print view View original

51–60 of 61 posts

Re: The Mathematics of Artificial Intelligence (2022)

#51

Post by mdp2021 »

alok-g wrote: post #48
mdp2021 wrote: post #22

Does anybody know of textbooks, or articles, material etc. that focus on the /consequences/ of the mathematics of ANNs? For example: why in this problem using more layers is more or less efficient than using bigger layers, or branching evolutions of the outcomes as opposed to intensive computation of a single flow etc. A reasoned summary of the tricks, in a way ("this works because of that").

Check this out: https://ai.facebook.com/blog/advancing-ai-theory-with-a-firs... HN discussion: https://news.ycombinator.com/item?id=27559017 My own review comment: https://news.ycombinator.com/item?id=27564506 Edit: Corrected the third link. Thanks mdp2021 for notifying me of the copy-paste error.

Thank you alok-g,

I would just correct the links:

-- article: https://arxiv.org/abs/2106.10165

-- book: https://arxiv.org/pdf/2106.10165

-- your summary: https://news.ycombinator.com/item?id=27564506 (the link to the post is that "under" the date. You posted two identical links here)

Re: The Mathematics of Artificial Intelligence (2022)

#52

Post by mdp2021 »

tomrod wrote: post #50
mdp2021 wrote: post #44

Earlier quoted context omitted.

I think in Greek 'neura' and 'neuron' are equivalent. The perception that '-on' indicate the single or ideal entity ("ion, electron, muon...") comes from Whewell and Faraday ('-on' ex "ienai": the "goer" particles) and fully postdates the use of 'neuron'. Edit: also see the comment from member martopix, nearby ( https://news.ycombinator.com/item?id=30988150 )

You have the correct etymology, but the convention is for "Neural Network" in ML and "Neuronal Network" in Neuroscience for we laymen.

As noted in the reference to the post from martopix :)

Re: The Mathematics of Artificial Intelligence (2022)

#54

Post by nancarrow »

akomtu wrote: post #39
mellosouls wrote: post #9

Let L(uy,y) = fy denote a parametric partial differential equation with y being a parameter from a high-dimensional parameter space Y ⊆ Rp and uy the associated solution in a Hilbert space H. After a high-fidelity discretization... These AI folks are clearly very clever, but they don't actually believe that's got anything to do with how human thinking works, right?

Whenever I see "Hilbert space" remarks in a paper that doesn't concern abstract math, I roll my eyes and imagine a chef who starts his youtube video with "Today we are going to cook a salad. For better precision, we are going to use knifes of triangular shape, three dimensional plates of circular shape, and the 100% dihydrogen of oxyde solution." All this only to obfuscate the fact that the entire recipe is just chop…

the subclass of "Reproducing kernel Hilbert spaces" form the basis for a large class of ML algorithms (kernel methods)

Re: The Mathematics of Artificial Intelligence (2022)

#55

Post by kvathupo »

plafl wrote: post #10
mg wrote: post #4

After a cursory glance, my feeling is that "The Mathematics of Neuronal Networks" would be a better title? In recent years, the term "Artificial Intelligence" is often used instead of "Neuronal Networks". I wouldn't be surprised if this will change again. If there is evidence that it will not and Neuronal Networks are for some reason the optimal medium for intelligence, I would love to read about it.

I like to explain that AI > ML > NN. Now, NNs are the ones getting results at computer vision and natural language, and more. I think most people would say that other ML approaches are computational statistics. The goalpost for AI keeps moving. If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gen…

Although it tends towards Deep Learning as opposed to AI, I highly recommend Bishop's Pattern Recognition and Machine Learning. It not only provides a solid Bayesian perspective, but also comments on the subtleties of applying theory. For the latter, its discussion of overparameterization in (I think?) the first chapter comes to mind.

[1] - https://www.microsoft.com/en-us/research/people/cmbishop/prm...

Re: The Mathematics of Artificial Intelligence (2022)

#56

Post by jjtheblunt »

akomtu wrote: post #35
plafl wrote: post #10

Earlier quoted context omitted.

I like to explain that AI > ML > NN. Now, NNs are the ones getting results at computer vision and natural language, and more. I think most people would say that other ML approaches are computational statistics. The goalpost for AI keeps moving. If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gen…

NN > ML. The proof is that nematode or fly, I dont remember, with its simple NN fully mapped, and still remaining a mystery how it works. ML, which is just a matrix multiplication at its core, is a laughably simplistic model of NN.

linear-algebraic matrix multiplication complicated by non-linear threshold discontinuities evaluators, no?

Re: The Mathematics of Artificial Intelligence (2022)

#57

Post by kvathupo »

akomtu wrote: post #35
plafl wrote: post #10

Earlier quoted context omitted.

I like to explain that AI > ML > NN. Now, NNs are the ones getting results at computer vision and natural language, and more. I think most people would say that other ML approaches are computational statistics. The goalpost for AI keeps moving. If you are truly interested in the math of AI I think PAC Bayes learning is more appropriate and your book is Understanding Machine Learning [1] (not an easy read). A more gen…

NN > ML. The proof is that nematode or fly, I dont remember, with its simple NN fully mapped, and still remaining a mystery how it works. ML, which is just a matrix multiplication at its core, is a laughably simplistic model of NN.

I think one can't compare deep learning to machine learning since they have different purposes. While there have been great strides in the interpretability of NNs, the analytic models of classical machine learning are favored from a computational and interpretability perspective. Conversely, if the relationships in our data are constantly evolving (e.g. the change in the stochastic process followed by a time-series from one interval to another), then NNs are more appropriate.

Re: The Mathematics of Artificial Intelligence (2022)

#58

Post by mellosouls »

martopix wrote: post #43
mellosouls wrote: post #9

Let L(uy,y) = fy denote a parametric partial differential equation with y being a parameter from a high-dimensional parameter space Y ⊆ Rp and uy the associated solution in a Hilbert space H. After a high-fidelity discretization... These AI folks are clearly very clever, but they don't actually believe that's got anything to do with how human thinking works, right?

It's literally called "the mathematics of ARTIFICIAL intelligence"

The important word is intelligence.

Re: The Mathematics of Artificial Intelligence (2022)

#59

Post by etiam »

dr_dshiv wrote: post #53

Has anyone followed Smolensky’s Harmonium approach? I think it is so cool that “goodness of fit” was based on “harmony”…

Have a search under the alternative name Restricted Boltzmann Machine if you haven't. Several interesting developments since the 80s.

Re: The Mathematics of Artificial Intelligence (2022)

#60

Post by dr_dshiv »

etiam wrote: post #59
dr_dshiv wrote: post #53

Has anyone followed Smolensky’s Harmonium approach? I think it is so cool that “goodness of fit” was based on “harmony”…

Have a search under the alternative name Restricted Boltzmann Machine if you haven't. Several interesting developments since the 80s.

Oh yeah well aware that it was the first restricted Boltzmann Machine. It minimizes energy, but that’s just harmony with a sign change. When Smolensky Hinton and Rummelhart collaborated, they decided to call it “goodness of fit.” So neural networks still optimize for harmony by another name. I think that’s so cool. The original Smolensky paper is worth reading. Good ideas.
Post reply on HN