Congratulations to Doctors Hopfield and Hinton! It's wonderful to see them both receiving such an esteemed prize, in recognition of the outstanding work they have both done.
Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
721–730 of 876 posts
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#722Earlier quoted context omitted.
CPA is very very widely used in experimental physics, so I don’t really think it belongs on this list.
Well yeah, so are neural nets. I just meant that these are engineering accomplishments, not scientific per se. Of course experimental science will often take advantage of cutting edge technology, including from computer science.
NNs have absolutely revolutionized systems biology (itself a John Hopfield joint, and the AlphaFold team are reasonably likely to get a Nobel for medicine and physiology, possibly as soon as 'this year') and are becoming relevant in all kinds of weird parts of solid-state physics (trained functionals for DFT, eg https://www.nature.com/articles/s41598-020-64619-8).
The idea that academic disciplines are in any way isolated from each other is nonsense. Machine learning is computer science; it's also information theory; that means it's thermodynamics, which means it's physics. (Or, rather, it can be understood properly through all of these lenses).
John Hopfield himself has written about this; he views his work as physics because _it is performed from the viewpoint of a physicist_. Disciplines are subjective, not objective, phenomena.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#723Earlier quoted context omitted.
I think there's some backlash against a google-able answer here. However, from memory the list of biggest awards for CS/Math are: Fields medal Abel prize Turing award Godel award
I don't think Geoff Hinton is in the running for a Fields medal[1] any more, unless they did what they did for Andrew Wiles and give him a "quantized" Fields medal. [1] You have to be under 40. https://www.fields.utoronto.ca/aboutus/jcfields/fields_medal...
Hinton was never in the running for a Fields medal since he never made a single contribution to the field of mathematics. His work is about empirical discoveries in CS.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#724Earlier quoted context omitted.
Or Obama’s peace prize.
It is different for Peace prize , it has always been political and different from day one , it is awarded by Norwegian noble committee which is appointed by the Norwegian parliament. All other prizes are awarded in Sweden by Swedish academy(for literature) , Royal Swedish Academy for sciences (physics and chemistry) , karolinska institute (physiology) are all professionally established organizations at the time of No…
Obama’s and Kissinger’s prizes are both disastrous for entirely different reasons. And I don’t mean anything partisan by that. Kissinger was awarded a peace prize for achieving a cease fire in the Vietnam war … a war in which he’d personally been responsible for some of the worst and most illegal excesses. Obama was awarded the peace prize for literally nothing. It was anticipatory, meant to urge him towards ending American wars in the Middle East (he escalated instead). The Nobel committee gets it wrong looking both forwards and backwards.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#725Earlier quoted context omitted.
The Nobel in Physics only goes with experimental discoveries, Peter Higgs didn't get his (deserved since the 70s) until the LHC directly observed the particle. I agree that Hopfield networks and Boltzmann machines are a surprisingly arbitrary choices. It is like they wanted to give a prize to someone for neural networks, but had to pick people from inside their own field to represent the development, which limited th…
I've never heard that it had to be tied to experimental discoveries. For example, Feynman got the prize for Feynman diagrams, path integrals and QED calculations. None of that directly tied to experimental work. It has definitely been awarded for both theoretical and experimental contributions throughout its history. Many theoretical physicists have received the prize for their conceptual breakthroughs, even without…
That was the reasoning given when Einstein won his prize for the photoelectric effect, not relativity (although the reasons were actually fairly complicated: https://www.advancedsciencenews.com/the-dramatic-story-behin...)
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#726Earlier quoted context omitted.
Well yeah, so are neural nets. I just meant that these are engineering accomplishments, not scientific per se. Of course experimental science will often take advantage of cutting edge technology, including from computer science.
NNs have absolutely revolutionized systems biology (itself a John Hopfield joint, and the AlphaFold team are reasonably likely to get a Nobel for medicine and physiology, possibly as soon as 'this year') and are becoming relevant in all kinds of weird parts of solid-state physics (trained functionals for DFT, eg https://www.nature.com/articles/s41598-020-64619-8 ). The idea that academic disciplines are in any way is…
My personal theory is that Demis and John will win the Chemistry prize for AlphaFold this year and that they decided to also award this one to help bolster the idea that ML is making fundamental improvements in academic science.
I would prefer if there was an actual Nobel Prize for Mathematics (not sure if the Fields would become that, or a new prize created).
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#727Earlier quoted context omitted.
Is this a widely accepted version of neural network history? I recognize Rosenblatt, Perceptron, etc., but I have never heard that Hopfield nets or Bolzmann machines were given any major weight in the history. The descriptions I have read were all mathematical, focusing on the computational graph with the magical backpropagation (which frankly is just memoizing intermediate computations). The descriptions also went o…
Bolzman machines were there in the very early days of deep learning. It was a clever hack to train deep nets layer wise and work with limited ressources. Each layer was trained similar to the encoder part of an autoencoder. This way the layerwise transformations were not random, but roughly kept some of the original datas properties. Up to here training was done without the use of labelled data. After this training s…
Boltzmann machines were there in the 1980s, and they were created on the basis of Hopfield nets (augmenting with statistical physics techniques, among other reasons to better navigate the energy landscape without getting stuck in local optima so much).
From the people dissing the award here it seems like even a particularly benign internet community like HN has little notion of ML with ANN:s before Silicon Valley bought in for big money circa 2012. And media reporting from then on hasn't exactly helped.
ANN:s go back a good deal further still (as the updated post does point out) but the works cited for this award really are foundational for the modern form in a lot of ways.
As for DL and backpropagation: Maybe things could have been otherwise, but in the reality we actually got, optimizing deep networks with backpropagation alone never got off the ground on it's own. Around 2006 Hinton started getting it to work by building up layer-wise with optimizing Restricted Boltzmann Machines (the lateral connections within a layer are eliminated from the full Boltzmann Machine), resulting in what was termed a Deep Belief Net, which basically did it's job already but could then be fine-tuned with backprop for performance, once it had been initialized with the stack of RBM:s. An alternative approach with layer-wise autoencoders (also a technique essentially created by Hinton) soon followed.
Once these approaches had shown that deep ANN:s could work though, the analysis showed pretty soon that the random weight initializations used back then (especially when combined with the historically popular sigmoid activation function) resulted in very poor scaling of the gradients for deep nets which all but eliminated the flow of feedback. It might have generally optimized eventually, but after way longer wait than was feasible when run on the computers back then. Once the problem was understood, people made tweaks to the weight initialization, activation function and otherwise the optimization, and then in many cases it did work going directly to optimizing with supervised backprop. I'm sure those tweaks are usually taken for granted to the point of being forgotten today, when one's favourite highly-optimized dedicated Deep Learning library will silently apply the basic ones without so much as being requested to, but take away the normalizations and the Glorot or whatever initialization and it could easily mean a trip back to rough times getting your train-from-scratch deep ANN to start showing results.
I didn't expect this award, but I think it's great to see Hinton recognized again, and precisely because almost all modern coverage is to lazy to track down earlier history than the 2010s, not least Hopfield's foundational contribution, I think it is all the more important that the Nobel foundation did.
So going back to the original question above: there are so many bad, confused versions of neural network history going around that whether or not this one is widely accepted isn't a good measure of quality. For what it's worth, to me it seems a good deal more complete and veridical than most encountered today.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#728Earlier quoted context omitted.
Someone changed the Wikipedia article today to call Hopfield a "physicist". Previously the article called him simply a scientist, because his main work wasn't limited to physics. I changed it back now, let's see if it holds up.
It’s ‘physicist’ again now.
User "ReyHahn" has changed it back to physicist. Justification: "he defines himself as physicist but he has worked i mnay fields"
https://en.wikipedia.org/w/index.php?title=John_Hopfield&dif...
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#729Earlier quoted context omitted.
machine learning is math, not physics. physics uses math, not the other way around. ML can be used in any field of science, not vice versa.
The actual process of computation, sure, but machine learning was born from physics-based methods and applications to understand complexity and disorder.
I think ML depended on math/statistics/computing much more than physics. It's much easier to see what Hopfield and Hinton did as inspired by mathematical models that were created to help study statistical mechanics. Which is fine and just shows how stochastic scientific discovery is.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#730I think this is the Royal Academy of Sciences way to admit that Physics as a research subject has ground to a halt. String theory suffocated theoretical high energy physics for nearly half a century with nothing to show for it, and a lot of other areas of fundamental physics are kind of done.
May I ask, very humbly, what are you credentials when making that assessment?