HN is the only place that helps laymen come to grips with information like this one. One need to read a lot of comments first and then jump to the link. Thanks guys/gals
Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
791–800 of 876 posts
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#792A lot of commentators here saying that while fundamental physics has possibly stalled, a lot of applied physics is still bustling with activity. Even buying that premise, is there anything in the applied physics fields that is comparable in impact to the neural network? It feels like the Nobel committee’s decision is an indictment on the lack of impact of modern physics. They had to stretch definitions and go into AI…
[deleted]
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#793AI is not Physics.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#794A lot of commentators here saying that while fundamental physics has possibly stalled, a lot of applied physics is still bustling with activity. Even buying that premise, is there anything in the applied physics fields that is comparable in impact to the neural network? It feels like the Nobel committee’s decision is an indictment on the lack of impact of modern physics. They had to stretch definitions and go into AI…
> Even buying that premise, is there anything in the applied physics fields that is comparable in impact to the neural network?
Why does it have to be? It's a different field. You would not expect the Nobel in chemistry to be awarded to Linus Torvalds, impact or no impact.
And the connection to physics is beyond tenuous here. The toy neural networks they cite in the document, including the Boltzmann machine, have very little to do with the power of ANNs to learn complex patterns that made such a splash recently. That is basically what the Bitter Lesson is all about. The interesting stuff does not arise from clever theory, but from practical tinkering and loooooots of compute.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#795A lot of commentators here saying that while fundamental physics has possibly stalled, a lot of applied physics is still bustling with activity. Even buying that premise, is there anything in the applied physics fields that is comparable in impact to the neural network? It feels like the Nobel committee’s decision is an indictment on the lack of impact of modern physics. They had to stretch definitions and go into AI…
The blue LED. Nobel Prize in 2014.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#796Earlier quoted context omitted.
That's the only reason I can think of for this - like, Alfred Nobel or some other dead donor set rules that they aren't allowed to change now. But I haven't seen anyone confirm it.
Alfred Nobel was the only donor, and his will spells out the rules: https://www.nobelprize.org/alfred-nobel/alfred-nobels-will/ In 1968, the Nobel Foundation accepted a donation from the Swedish central bank to establish a prize in economy, but in hindsight that was a pretty bad idea, and the probability of them accepting future donations to establish prizes in other fields is very slim.
They STILL don't call it a nobel prize. That's something.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#797I asked ChatGPT o1-preview with cutt-off October 2023 and it was not enthusiastic about the idea and said medicine would be closer.
ChatGPT 40 wasn't keen about it either.
Perplexity.ai told me it is what it is, it's a great idea and it's fine with it.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#798Earlier quoted context omitted.
Then they shouldn't be trusted to give awards in an area they are not experts in.
Impact is hard to quantify. There have been several occasions where someone who very well deserved a Nobel prize didn't get one. There are all kinds of reasons. Given he is mentioned in the reasoning he was probably considered. We can't know the reason he did not get the prize. I recently watched this quite video on the subject: https://youtu.be/zS7sJJB7BUI?feature=shared and found it quite enjoyable.
A funny one is that there was so much objection to General Relativity that they compromised by giving Einstein a Nobel for his work on quantum mechanics.
His acceptance speech was about general relativity.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#799Earlier quoted context omitted.
Would that be equivalent to Weight matrice parameters?
Ish, take a look at the curves of the spiking neural network function, they are very different from the deep learning nets. When we "model" biological neural nets in code, we are essentially coming up with a mathematical transfer function that can replicate the chemical gradient changes and electrical impulses of a real neuron. Imagine playing a 3D computer game like Minecraft, the physics is not perfect but they are…
Thanks this is very insightful.
Would the multi-head attention (Wv) not be equivalent to the chemical gradient changes?
(there are multiple matrices in multi-head attention, one for each attention head and what I imagine would be the equivalent of different gradients
This allows each attention head to learn different representations and focus on different aspects of the input sequence.)
And then the output produced after applying the concatenated (W0 or output projection), be equivalent to the different electrical outputs such as the spikes and passed to the next neuron equivalent or attention head?
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#800A lot of commentators here saying that while fundamental physics has possibly stalled, a lot of applied physics is still bustling with activity. Even buying that premise, is there anything in the applied physics fields that is comparable in impact to the neural network? It feels like the Nobel committee’s decision is an indictment on the lack of impact of modern physics. They had to stretch definitions and go into AI…
EUV lithography? Probably has had a bigger impact on AI today than Hopfield networks and Boltzmann machines combined. And also: it’s actually physics.