Live data from Hacker News

It took me 10 years to understand entropy

cantorsparadise.com

91–100 of 295 posts

Re: It took me 10 years to understand entropy

#91

I don't understand entropy and this article did not change it. The issue I take is with the definition of "the most likely state". Think of a series of random bits that can be either 0 or 1 with equal probability. How likely is it that they are all 0 or all 1? Not very likely. There is exactly one configuration. How likely is it that they have a specific configuration of 0 and 1? Equally likely. All states are equall…

Not a physicist either, and I don't claim to understand entropy that well either but maybe it would help to consider that entropy may not be a universal variable of systems in the universe.

I think you should rather consider it as a mathematical construct that applies to some systems where the microscopic quantities are well defined, and where the 'averaging' that we can observe is also well defined. So if you look at thermodynamics, entropy is well defined, but you may be totally right, that what we call "microscopic states" in a gas can be broken down further in elementary particles, that may or may not behave in quantic ways, and what not, and counting the micro-states considering the elementary particles is a whole different game.

But it doesn't really matter. What matters is that at the scale we're at and with the microscopic/macroscopic relation that's defined, entropy works. The calculations that give some numbers to entropy show that it looks like entropy cannot decrease. They call it a universal principle of thermodynamics, because there is nothing (to my understanding), that explains it microscopically.

And it works for a variety of situation in physics, such that it seems that it's a universal property of nature. But it's mostly mathematical. It seems to say that "given a system we know everything about, there is no way to go to a system that has some unknown things to us".

Anyways. I mostly wrote this to see if I could articulate it to myself, hopefully it helps you as well.

Re: It took me 10 years to understand entropy

#92

but is it not hubris to think that we really know much about the origin and outcome of the universe? is it wise to make decisions based on this modicum of knowledge that we currently have regarding thermodynamics and the universe? I suspect that the scientists of a trillion years from now will know a lot more than we do know...so, I don't really much that much confidence in current pronouncements regarding the beginn…

The models we have work and are relatively parsimonious. It would be silly to assume they are final but equally silly to shy away from using them for obvious reasons.

The modernization of physics also means we have outlines for what theories should look like, so even if our current theories are wrong we can still use the principles of (say) symmetry and information to constrain future work.

Re: It took me 10 years to understand entropy

#93

The author mentions Boltzman brains and that a human body could theoretically spontaneously form out of particles given a long enough time span. Of course, nothing like this can ever happen. It’s the fallacy of thinking infinite time means infinite possibilities.

Perhaps it's actually correct, but our intuitions about ridiculously long periods of time aren't good. Note that heat death is in ~10¹⁰⁰ years, whereas this Boltzman body would take ~10^(10⁶⁹) years. That second time period is literally incomprehensible. So we think, of course a fully formed human body wouldn't appear; in practice, that's not how it actually works; the fact that it appears possible is at best a mathematical artifact, not reality. But we're talking about a timescale that's not just longer than the age of the universe, or than the total lifespan of the universe, not just orders of magnitude longer than those times, but on a completely different scale. Given that, I think we need to toss out those intuitions.

As to the author's last question of whether such a thing even makes sense at all given those time scales, I don't see why not. After all, once the universe reaches heat death, as far as we know nothing from the outside is going to come along and garbage collect it, so why couldn't it last for an arbitrary/infinite number of years? And compared to that, ~10^(10⁶⁹) years, or ~10^(10^(10⁵⁶)) years, or whatever, is nothing.

Re: It took me 10 years to understand entropy

#94
post #56
post #25

One aspect of entropy that I always find counterintuitive is that unlike mass, charge, etc. it is not a physical quantity. In fact, from the point of view of an experimenter with perfect information about a physical system, the entropy of the system is exactly conserved over time (as made precise by Liouville's Theorem). The Second Law survives in this setting only in the most trivial sense that a constant function d…

Entropy in thermodynamics is a statistical effect which acts like a "force" because of the immense number of particles and sub-states in play. A perfect simulation of gas particles bouncing in a two-chamber system will result in the "pressure" equalising because that is overwhelmingly the most likely state to end up in. To be honest, I hadn't heard of Louisville's Theorem before but it doesn't seem to imply what you'…

Liouville's Theorem does indeed seem to imply that entropy doesn't change:

https://physics.stackexchange.com/questions/202522/how-is-li...

Re: It took me 10 years to understand entropy

#95
post #25

One aspect of entropy that I always find counterintuitive is that unlike mass, charge, etc. it is not a physical quantity. In fact, from the point of view of an experimenter with perfect information about a physical system, the entropy of the system is exactly conserved over time (as made precise by Liouville's Theorem). The Second Law survives in this setting only in the most trivial sense that a constant function d…

> One aspect of entropy that I always find counterintuitive is that unlike mass, charge, etc. it is not a physical quantity. Those physical quantities might be intuitive, but as a physicist Brian Greene once wrote, no one really knows what mass is. We only know that mass bends space-time curve, hence gravity.

> no one really knows what mass is. We only know that mass bends space-time curve, hence gravity.

Mass is much better understood by its role in inertia. Basically mass is the amount of energy you need to exchange with a thing to change its current speed. This observation works from Newtonian mechanics to QM and GR as well.

Now, why do things have mass? The famous E=mc² explains this for most things: they have mass because something inside them has potential or kinetic energy. This works all the way down to the atomic level - the mass of a proton for example is almost entirely explained by the potential energy of the quarks being held together in a small volume; the total mass of the quarks themselves is only a small fraction of that. Now, the mass of the elementary particles is somewhat more complicated, but the Standard Model does have explanations for those - symmetry breaking for fermions, and the Higgs mechanism for the massive bosons.

The next mystery is: why is inertial mass equal to gravitational mass? GR has essentially explained this, by showing that acceleration is equivalent to gravitational attraction depending on your frame of reference.

So overall, I'm not sure what Brian Greene means by that - mass is at least as well understood as other basic properties of particles (charge, spin, color charge).

This lecture by Leonard Susskind explains most of these things about mass in a way I found easy to follow:

https://www.youtube.com/watch?v=JqNg819PiZY

Re: It took me 10 years to understand entropy

#96
post #59

I thought entropy (in the Shannon sense) was a property of discrete and finite probability distributions. It's essentially a measure of how random a sample from such a probability distribution is. Notably, continuous probability distributions don't have meaningful entropy (or in some sense, their entropy is always infinite). It's worth considering the similarities and differences between entropy and standard deviatio…

> I thought the 2nd law of thermodynamics was saying that with incomplete knowledge, the probability distribution of possible states becomes more and more spread out as time goes on. It's almost a limit to how you can make predictions or simulations of physics when the initial state of the system is not fully known. Equivalently, it's a banal statement about chaos in the sense of chaos theory.

I'm not sure I understand what do you mean by "as time goes on". Classical thermodynamical entropy is defined for a system in equilibrium and it doesn't change with time. It changes when you do things to the system.

Re: It took me 10 years to understand entropy

#97
post #59

I thought entropy (in the Shannon sense) was a property of discrete and finite probability distributions. It's essentially a measure of how random a sample from such a probability distribution is. Notably, continuous probability distributions don't have meaningful entropy (or in some sense, their entropy is always infinite). It's worth considering the similarities and differences between entropy and standard deviatio…

> The only thing I don't get is how physicists get around the discrete and finite restriction.

Actually, they don't! When you start doing the math about states in a quantum sense (i.e. statistical mechanics), the basic premise is that the available range of states _is_ discrete. Particles are quantized - so they can only possess certain allowable discrete energy levels. The broader laws of thermodynamics fall out of that and appear to be continuous as you scale up to the macro world across a huge number of microstates.

Re: It took me 10 years to understand entropy

#98
post #96
post #59

I thought entropy (in the Shannon sense) was a property of discrete and finite probability distributions. It's essentially a measure of how random a sample from such a probability distribution is. Notably, continuous probability distributions don't have meaningful entropy (or in some sense, their entropy is always infinite). It's worth considering the similarities and differences between entropy and standard deviatio…

> I thought the 2nd law of thermodynamics was saying that with incomplete knowledge, the probability distribution of possible states becomes more and more spread out as time goes on. It's almost a limit to how you can make predictions or simulations of physics when the initial state of the system is not fully known. Equivalently, it's a banal statement about chaos in the sense of chaos theory. I'm not sure I understa…

I don't think statistical mechanics entropy is limited in this way. I think the (incorrect? oversimplified?) definition given in the article is only valid under the conditions you've given. But I'm not sure.

Re: It took me 10 years to understand entropy

#99
post #98
post #96

Earlier quoted context omitted.

> I thought the 2nd law of thermodynamics was saying that with incomplete knowledge, the probability distribution of possible states becomes more and more spread out as time goes on. It's almost a limit to how you can make predictions or simulations of physics when the initial state of the system is not fully known. Equivalently, it's a banal statement about chaos in the sense of chaos theory. I'm not sure I understa…

I don't think statistical mechanics entropy is limited in this way. I think the (incorrect? oversimplified?) definition given in the article is only valid under the conditions you've given. But I'm not sure.

Then it maybe depends on what you meant by "the 2nd law of thermodynamics".

Re: It took me 10 years to understand entropy

#100

I don't understand entropy and this article did not change it. The issue I take is with the definition of "the most likely state". Think of a series of random bits that can be either 0 or 1 with equal probability. How likely is it that they are all 0 or all 1? Not very likely. There is exactly one configuration. How likely is it that they have a specific configuration of 0 and 1? Equally likely. All states are equall…

Have you looked at Huffman coding? I'd recommend the (free) book by David MacKay, it is secretly the "hackers guide to thermodynamics".

http://www.inference.org.uk/mackay/itila/book.html

Post reply on HN