I suppose LeCun and Bengio are way too young for a Nobel prize these days[0] [0] https://www.ageing.ox.ac.uk/images/fig01.jpeg
My guess is that Ilya would be more likely to get it at some point in the future than the two of them combined.
Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
821–830 of 876 posts
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#822Earlier quoted context omitted.
He was probably considered since he is mentioned in the reasoning paper, still it could be one of those unfortunate omissions in the nobel history since those deciding the prize might have a hard time to measure impact.
Then they shouldn't be trusted to give awards in an area they are not experts in.
Don't indulge in meritocracy. It's essentially a "prize".
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#823Earlier quoted context omitted.
Those two buckets only contain the work in physics that have a sustained presence in popular media. But take gravitational wave astronomy as a counterexample. It doesn't make it into the news much, but I'm pretty sure the entire field is less than ten years old.
Yes, good example. They already got a Nobel prize, one of the quickest in history. This is the kind of physics we might need more of.
So then you could argue that that's what the folks who are smashing particles and building new models are trying to get to.
Smashing particles helps test existing theories and hypotheses. We do it with particle accelerators because that's one of the ways of getting to the uncharted territory, which is where you need to be if you want to push the boundaries. And maybe remember that the sexy stuff that makes it into the news isn't the whole of the thing. The LHC is also, for example, doing practical climate science: https://en.wikipedia.org/wiki/CLOUD_experiment
And building new mathematical models is part of figuring out how to make sense of observations that don't quite fit the current models. That is a messy process, and I think that our retrospective perspective on what that process is like might be colored by survivor bias. We remember Einstein and his theory of special relativity. We mostly don't remember the preceding few decades' worth of other attempts by other physicists to resolve conflicts between existing non-unified theories (in this case Newton's and Maxwell's models) or making sense of things like the Michelson-Morley experiment. I don't really know that history myself, but I would not at all be surprised if many of those efforts were also having trouble figuring out how to produce testable hypotheses.
And also, big picture, I think that it's important for any lover of science to remember to celebrate the entire enterprise, not just its headline successes. Expecting consistent results is tacitly expecting scientists to have some way of knowing ahead of time which avenues of inquiry will be most fruitful. If we had access to an oracle that could tell us that, we wouldn't actually need science anymore.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#824Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#825Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#826Earlier quoted context omitted.
To be perfectly honest, I'm not really sure the physics community being demoralized by this prize award is a really negative thing. I think many aspects of HEP and other areas have stagnated, requiring exponentially more money for marginal gains (I don't have a problem with LIGO or other large projects, but "we found another high energy particle consistent with the standard models" not so much). Perhaps this prize wi…
More fodder for a Sabine Hossenfelder video please
Sabine hates LIGO.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#827Earlier quoted context omitted.
I would say that's an overly simplistic view. The only way we ever obtain testable things is by working on things which are not currently testable.
It used to be that there was some experimental result or other phenomena that required explanation which lead to a theoretical model that could be tested. That worked very well. Now there’s some theoretical considerations that leads to a theoretical model that can’t be tested. It didn’t work for Aristotle and it doesn’t work for string theorists (and similar).
Why doesn't this experimental result count as requiring explanation?
We know (for example) silver atoms have mass, and that massive objects exert gravity (which we understand as warping of space-time according to GR).
We know that we can put silver atoms in quantum superpositions of being in different positions (for example in a sequential Stern-Gerlach type experiment).
We have (essentially) absolutely no theoretical understanding of what is going on to space-time when a thing with mass is in such a superposition. Quantum mechanics does not successfully model gravity, and general relativity contains no superpositions, so the situation is completely beyond our theoretical understanding. This isn't a theoretical consideration, this is something real that you can do in an undergrad physics lab experiment pretty easily.
Now the problem is that the models we have developed so far to deal with this situation turned out to be (wildly) too difficult for us to test. I think it is very far from clear that the Oppenheim & co model falls into this category - imo its completely reasonable for them to be spending theoretical effort working out what is needed to test their model.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#828I wonder if they ever gave a Physics Nobel to a person who held a patent! People like Graham Bell never got recognized by the Nobel people. I get the impression that Physics Nobel prizes were more or less given only to University professors. They didnt seem to particularly care for people with grease on their hands
I don't really know what I'm talking about, but weren't there like 9 Nobel prizes awarded to Bell Labs engineers for physics? One of which (I think) being the invention of the transistor, which presumably had a patent.
You make several good points. However I'm not sure if Bell Labs people were different from university people in terms of their academic background. All three, Shockley, Bardeen and Brattain were Physics PhDs, two of them from elite universities. I was trying to say that brilliant engineers do not disproportionately figure in the list of Physics Nobel prize. In fact they are hardly to be seen.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#829Earlier quoted context omitted.
> Hopfield networks and Boltzmann machines Think of this as a Nobel prize for systems physics – essentially "creative application of statistical mechanics" – and it makes a lot more sense why you'd pick these two. (I am a mineral physicist who now works in machine learning, and I absolutely think of the entire field as applied statistical mechanics; is that correct? Yes and no: it's a valid metaphor.)
Having studied control engineering, it feels like ML is control theory + optimization all the way down :) I love how folks from different backgrounds can interpret it in so many ways.
It’s also signal processing.
Re: Nobel Prize in Physics awarded to John Hopfield and Geoffrey Hinton [pdf]
#830Earlier quoted context omitted.
Ish, take a look at the curves of the spiking neural network function, they are very different from the deep learning nets. When we "model" biological neural nets in code, we are essentially coming up with a mathematical transfer function that can replicate the chemical gradient changes and electrical impulses of a real neuron. Imagine playing a 3D computer game like Minecraft, the physics is not perfect but they are…
Thanks this is very insightful. Would the multi-head attention (Wv) not be equivalent to the chemical gradient changes? (there are multiple matrices in multi-head attention, one for each attention head and what I imagine would be the equivalent of different gradients This allows each attention head to learn different representations and focus on different aspects of the input sequence.) And then the output produced a…
¯\_(ツ)_/¯