Earlier quoted context omitted.
Von Neumann was the king of kings
He's a certified Martian: https://en.wikipedia.org/wiki/The_Martians_(scientists) .
I was hoping the Wikipedia might explain why this might have been.
What Is Entropy?
121–130 of 221 posts
Re: What Is Entropy?
#122Earlier quoted context omitted.
So the only thing you need to know about entropy is that it's a real-valued number you can associate with a probability distribution ? And that's it? I disagree. There are several numbers that can be associated with probability distribution, and entropy is an especially useful one, but to understand why entropy is useful, or why you'd use that function instead of a different one, you'd need to know a few more things…
In particular, the expectation (or variance) of a real-valued random variable can also be seen as "a real-valued number you can associate with a probability distribution". Thus, GP's statement is basically: "entropy is like expectation, but different".
[deleted]
Re: What Is Entropy?
#123If I would write a book with that title then I would get to the point a bit faster, probably as follows. Entropy is just a number you can associate with a probability distribution. If the distribution is discrete, so you have a set p_i, i = 1..n, which are each positive and sum to 1, then the definition is: S = - sum_i p_i log( p_i ) Mathematically we say that entropy is a real-valued function on the space of probabi…
So the only thing you need to know about entropy is that it's a real-valued number you can associate with a probability distribution ? And that's it? I disagree. There are several numbers that can be associated with probability distribution, and entropy is an especially useful one, but to understand why entropy is useful, or why you'd use that function instead of a different one, you'd need to know a few more things…
Of course that is not my statement. See all my other replies to identical misinterpretations of my comment.
Re: What Is Entropy?
#124Re: What Is Entropy?
#125Earlier quoted context omitted.
He was really brilliant, made contributions all over the place in the math/physics/tech field, and had a sort of wild and quirky personality that people love telling stories about. A funny quote about him from a Edward “a guy with multiple equations named after him” Teller: > Edward Teller observed "von Neumann would carry on a conversation with my 3-year-old son, and the two of them would talk as equals, and I somet…
Are there many von-Neumann-like multidisciplinaries nowadays? It feels like unless one is razor sharp fully into one field one is not to be treated seriously by those who made careers in it (and who have the last word on it).
There have been a very small number of thinkers as publicly accomplished as von Neumann ever. One other who comes to mind is Carl F. Gauss.
Re: What Is Entropy?
#126If I would write a book with that title then I would get to the point a bit faster, probably as follows. Entropy is just a number you can associate with a probability distribution. If the distribution is discrete, so you have a set p_i, i = 1..n, which are each positive and sum to 1, then the definition is: S = - sum_i p_i log( p_i ) Mathematically we say that entropy is a real-valued function on the space of probabi…
Don’t forget it’s the only measure of the arrow of time.
One could also say that it’s just a consequence of the passage of time (as in getting away from a boundary condition). The decay of radioactive atoms is also a measure of the arrow of time - of course we can say that’s the same thing.
CP violation may (or may not) be more relevant regarding the arrow of time.
Re: What Is Entropy?
#127Earlier quoted context omitted.
Thanks for defining it rigorously. I think people are getting offended on John Baez's behalf because his book obviously covers a lot more - like why does this particular number seem to be so useful in so many different contexts? How could you have motivated it a priori? Etcetera, although I suspect you know all this already. But I think you're right that a clear focus on the maths is useful for dispelling misconcepti…
Misconceptions about entropy are misconceptions about physics. You can’t dispell them focusing on the maths and ignoring the physics entirely - especially if you just write an equation without any conceptual discussion, not even mathematical.
I didn't say to only focus on the mathematics. Obviously wherever you apply the concept (and it's applied to much more than physics) there will be other sources of confusion. But just knowing that entropy is a property of a distribution, not a state, already helps clarify your thinking.
For instance, you know that the question "what is the entropy of a broken egg?" is actually meaningless, because you haven't specified a distribution (or a set of micro/macro states in the stat mech formulation).
Re: What Is Entropy?
#128Earlier quoted context omitted.
Thanks, that's an interesting perspective. It also highlights one of the weak points in the concept, I think, which is that this is only a tool for updating beliefs to the extent that the underlying probability space ("ontology" in this analogy) can actually "model" the phenomenon correctly! It doesn't seem to shed much light on when or how you could update the underlying probability space itself (or when to change y…
Couldn't you just add a control (PID/Kalman filter/etc) to coverage on a stability of some local "most" truth?
Could you elaborate? To be honest I have no idea what that means.
Re: What Is Entropy?
#129Am I only one that can't download the pdf, or is the file server down? I can see the blog page but when I try downloading the ebook it just doesn't work..
If the file server is down.. anyone could upload the ebook for download?
Re: What Is Entropy?
#130Earlier quoted context omitted.
He's a certified Martian: https://en.wikipedia.org/wiki/The_Martians_(scientists) .
I was hoping the Wikipedia might explain why this might have been.