Live data from Hacker News

Entropy: A little understood concept in physics [video]

youtube.com

61–70 of 178 posts

Re: Entropy: A little understood concept in physics [video]

#61
While this is a reasonable historical explanation of entropy, and explains that we don't gain net energy from the sun, it still misses the mark on what entropy is now known to be.

Entropy isn't a property of an object, or a system or things in physics. Entropy is a property of our _description_ of systems. More precisely it is a measure of how poorly a given specification of a physical system is, i.e. given description of a systems, typically the pressure / volume / temperature of a gas or whatnot, how many different physical systems correspond to such a description.

In particular, _thermodynamic entropy is Shannon entropy_.

In the case where the description of state specifies a volume of phase space wherein a physical state lies within, then the entropy is the logarithm of the volume of this fragment of phase space. If we take this collection of states and see how they evolve in time, then Liouville’s theorem says the volume of phase space will remain constant.

If we want to build a reliable machine, i.e. an engine, that can operate in any initial state that is bounded by our description, and ends up win a final state bounded by some other description, well, in order for this machine to preform reliably, the volume of the final description needs to be greater than the volume of the description of the initial state. Otherwise, some possible initial states will fail to end up in the desired final state. This is the essence of the second law of thermodynamics.

I want to emphasis this: entropy exists in our heads, not in the world.

E.T. Jaynes illustrated this "5. The Gas Mixing Scenario Revisited" in https://www.damtp.cam.ac.uk/user/tong/statphys/jaynes.pdf where two imaginary variants of Argon gas are mixed together. If one engineer is ignorant of the different variants of Argon gas, it is impossible to extract work from the gas, but armed with knowledge of the difference (which must be exploitable otherwise they wouldn't actually be different) work can be extracted.

Knowledge _is_ power.

Taking an extreme example, suppose we have two volumes of gas at different volumes / pressures / temperature. We can compute how much work can be extracted from those gases.

But, suppose someone else knows more than just the volume / pressure / temperature of these gases. This someone happens to know the precise position and velocity of every single molecule of gas (more practically they know the quantum state of the system). This someone now gets to play a the role of Maxwell's demon and separate all the high velocity and low velocity molecules of each chamber, opening and closing a gate using their perfect knowledge of where each particle is at each moment in time. From this they can now extract far more work than the ignorant person.

In both cases the gas was identical. How much useful work one can extract depends on how precise one's knowledge of the state of that gas is.

Re: Entropy: A little understood concept in physics [video]

#62

The part that was new to me was the bit about how a space full of life tends toward more entropy faster than the same amount of space without life. Like the best ideas, it’s simple and makes sense if you think about it, but it’s still a really interesting framing that the complex machinery of life is really just the most efficient “entropy converter”. If there’s something about the arrow of time that speeds towards t…

Does Einstein’s model of time care about entropy. In other words if there are 2 regions, one where entropy is increasing at that time and one where it isn’t as much, does it affect time?

Re: Entropy: A little understood concept in physics [video]

#63

Earlier quoted context omitted.

Not really - look up “heat death” of the universe.

The "heat death" of the universe is a concept that deserves to die. The second principle of thermodynamics is true only if you ignore gravity. In the presence of gravity, systems tend to go towards lower entropy, just see how a planetary system can form out of a gas cloud.

> In the presence of gravity, systems tend to go towards lower entropy, just see how a planetary system can form out of a gas cloud.

This isn't correct: the entropy (and energy) of the gas cloud goes decreases as it collapses, but the entropy of its surroundings increases faster as it radiates.

Re: Entropy: A little understood concept in physics [video]

#64
post #39

Earlier quoted context omitted.

> Do we even have a word for the opposite of entropy? I've always referred to the inverse as "information" or "order".

"information" is not a good term for the opposite. Entropy is a well defined concept in information theory (and can be connected to the physical concept). More entropy means more information, not less.

I agree. Order is a better term to describe it. Predictability. More entropy = more total possible states the system can be in.

I tend to use "information" to refer to a statistically-significant signal or data that the application/business can practically utilize. This is definitely not the same as the strict information theoretical definition.

Re: Entropy: A little understood concept in physics [video]

#65

Entropy only made sense when I learned it from the perspective of statistical thermodynamics. It's a very programmerly understanding, IMHO, and it's quite intuitive. EXCEPT that the language used is ridiculous: grand canonical ensemble indeed! Anyway, the idea that a system can be in some number of specific states, and that equilibrium is that unique situation where the number of possible specific states is at its ma…

Entropy is mathematical force to be honest.

[dead]

Re: Entropy: A little understood concept in physics [video]

#66

Sabine Hossenfelder also had a video recently on entropy: > I don't believe the 2nd law of thermodynamics. https://www.youtube.com/watch?v=89Mq6gmPo0s

What I really like about this explanation is that it highlights the fact that entropy is not a natural property of the physics system: entropy is only defined with respect to some coarse-graining operation applied by an imperfect observer of the system. So as Sabine points out it seems we should really be talking about multiple different entropies, each of which corresponds to a different mechanism for coarse-graining microstates into macrostates, with each different entropy changing at different rates depending on the coarse-graining mechanism and physical system. (And in particular, God observing the universe would not see entropy change at all; even if there were uncertainty in the initial conditions of the universe, God would see that uncertainty perfectly propagated with no loss of information, in a way made precise by Liouville's Theorem.)

But even this is not the full story, because I can take a mass-spring network, and no matter how I choose to coarse-grain it, I will not see the entropy corresponding to that coarse-graining increase, because the trajectory of a mass-spring system is periodic. Entropy increase requires that the system is ergodic with respect to the chosen coarse-graining operation, i.e. that over long times the trajectory visits the coarse-grained states in a "random" and uniform way. It's not at all obvious to me why the dynamics of particles bouncing around in a box have this property, and particles attached in a mass-spring network do not; and neither the Sabine nor the Veritaserum videos address this or why we should expect all practical real-world physical systems to be ergotic with respect to practical coarse-graining mechanisms.

Re: Entropy: A little understood concept in physics [video]

#67

Earlier quoted context omitted.

The "heat death" of the universe is a concept that deserves to die. The second principle of thermodynamics is true only if you ignore gravity. In the presence of gravity, systems tend to go towards lower entropy, just see how a planetary system can form out of a gas cloud.

> In the presence of gravity, systems tend to go towards lower entropy, just see how a planetary system can form out of a gas cloud. This isn't correct: the entropy (and energy) of the gas cloud goes decreases as it collapses, but the entropy of its surroundings increases faster as it radiates.

Is that a fact? Or just a hypothetical way that could save the second principle?

Re: Entropy: A little understood concept in physics [video]

#68
post #54

Entropy only made sense when I learned it from the perspective of statistical thermodynamics. It's a very programmerly understanding, IMHO, and it's quite intuitive. EXCEPT that the language used is ridiculous: grand canonical ensemble indeed! Anyway, the idea that a system can be in some number of specific states, and that equilibrium is that unique situation where the number of possible specific states is at its ma…

I took a stat thermo class and it was basically all about entropy, which was expresed as ln W- the log of the number of ways (permutations) that a system can be ordered, which gives a convenient denominator when calculating the probability of a specific permutation. Here's the professor's book, which was still only in latex form when we took the class: https://www.amazon.com/Molecular-Driving-Forces-Statistical-...

yes, there are lots of quantitative details. I wanted to emphasize the key qualitative concept, from which the others can derive. In a similar way you can derive all of special relativity, and approach an intuition about the strangeness of spacetime, starting with only two ideas: the laws of physics are the same in all reference frames; the speed of light is constant. I prefer to start there and derive e.g. Lorentz factors than start with the mathy stuff.

Re: Entropy: A little understood concept in physics [video]

#69

Earlier quoted context omitted.

> In the presence of gravity, systems tend to go towards lower entropy, just see how a planetary system can form out of a gas cloud. This isn't correct: the entropy (and energy) of the gas cloud goes decreases as it collapses, but the entropy of its surroundings increases faster as it radiates.

Is that a fact? Or just a hypothetical way that could save the second principle?

It's a fact. See for instance https://arxiv.org/pdf/0907.0659.pdf

Re: Entropy: A little understood concept in physics [video]

#70
I don't think this delivers the intuition in a simple manner. It's also not fully correct. People explain entropy in over complicated ways and even in this thread many people explaining it don't get it. There is a simple way to think about this and I guarantee if you read my explanation you'll understand it more.

In essence what you need to realize is that entropy is just a label for an aspect of probability.

Things tend to become disordered over time because disordered states are more probable then ordered states. Entropy is thus simply phenomenon of probability... of things moving from a low probability state to a high probability state. That's it.

That's really all there is to it. That's all you need to digest, all the complicated math and explanations are all just surrounding the above concept.

Entropy is just a high level abstraction of probability. It just allows you to explain things without the intuition of probability bogging you down. For example, explaining life in terms of probability is harder to grasp as it's akin to rolling 10 dice and having all the dice roll a 6.

Post reply on HN