Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is entropy? A measure of just how little we know

quantamagazine.org

Post reply on HN
Print view View original

91–100 of 169 posts

Re: What is entropy? A measure of just how little we know

#91

Post by kergonath »

notfed wrote: post #77
kergonath wrote: post #61

Earlier quoted context omitted.

> The quantities we choose to measure are dictated by our psychological and physiological limitations. No. The enthalpy changes measured by a calorimeter are not dependent on our psychological limitations. > The volume of a container is not objectively defined. Yes, it is, for any reasonable definition of "objective". We know how to measure lengths, we know how they change when we use different frames of reference so…

Sometimes I wish HN had merit badges. Or if you like, a device to measure the amount of information contained within a post.

I am not sure it would help, I think it would just enhance groupthink. I like how you need to write something obviously stupid or offensive for the downvotes to have a visible effect and that upvotes have no visible effect at all. (Yes, it changes ranking, but there are other factors). People are less prejudiced when they read the comment than if they see that it is already at -3 or +5.

Yes, it means that some posts should be more (or less) visible than they are but overall I think it’s a good balance.

Besides, I am not that interested in the absolute amount of information in a post. I want information that is relevant to me, and that is very subjective :)

Re: What is entropy? A measure of just how little we know

#92

Post by kergonath »

kgwgk wrote: post #69
kergonath wrote: post #64

Earlier quoted context omitted.

By the look of it, it is another misguided attempt to apply information theory concepts to thermodynamics. Entropy as information is seductive because that way we think we can understand it better, and it looks like it works. But we need to be careful because even though we can get useful insights from it (like Hawking radiation) it’s easy to reach unphysical conclusions. > How is that not right in physics? Why would…

That misguided attempt has resulted in full graduate courses and textbooks. Maybe one look is not all that it takes to fully assess its worthiness. https://www.amazon.com/Microphysics-Macrophysics-Application... https://arxiv.org/pdf/cond-mat/0501322

Plenty of bad stuff made its way into textbooks, I saw some really dodgy stuff at uni. And for every uncontroversial statement we can find a textbook that argues that it is wrong. Sorting the good from the bad is the main point of studying science and it is not easy. What is also important is that approaches that can work in some field or context might be misleading or lead to wrong outcomes in others. Information theory is obviously successful and there is nothing fundamentally wrong with it.

Where we should be careful is when we want to apply some reasoning verbatim to a different problem. Sometimes it works, and sometimes it does not. Entropy is a particularly good example. It is abstract enough to be mysterious for a vast majority of the population, hence why these terribly misleading vulgarisation articles pop up so often. Thinking of it in terms of information is sometimes useful, but going from information to knowledge is a leap, and then circling back to Physics is a bit adventurous.

Re: What is entropy? A measure of just how little we know

#93

Post by kgwgk »

kergonath wrote: post #92
kgwgk wrote: post #69

Earlier quoted context omitted.

That misguided attempt has resulted in full graduate courses and textbooks. Maybe one look is not all that it takes to fully assess its worthiness. https://www.amazon.com/Microphysics-Macrophysics-Application... https://arxiv.org/pdf/cond-mat/0501322

Plenty of bad stuff made its way into textbooks, I saw some really dodgy stuff at uni. And for every uncontroversial statement we can find a textbook that argues that it is wrong. Sorting the good from the bad is the main point of studying science and it is not easy. What is also important is that approaches that can work in some field or context might be misleading or lead to wrong outcomes in others. Information th…

Plenty of bad stuff makes its way into hackernews comments as well. Saying that “it could be wrong” doesn’t really support the “it’s wrong” claim, does it?

Re: What is entropy? A measure of just how little we know

#94

Post by kergonath »

oh_my_goodness wrote: post #71
kergonath wrote: post #64

Earlier quoted context omitted.

By the look of it, it is another misguided attempt to apply information theory concepts to thermodynamics. Entropy as information is seductive because that way we think we can understand it better, and it looks like it works. But we need to be careful because even though we can get useful insights from it (like Hawking radiation) it’s easy to reach unphysical conclusions. > How is that not right in physics? Why would…

"We need to be careful", agreed. "It's not mundane", agreed. (It's mundane in information theory because that's how they define entropy.) "It's [...] not right" (from your first comment), can you give/link a specific physical example? It would be very cool to have a clear counterexample.

> can you give/link a specific physical example?

About the lack of subjectivity of the states? If we consider any bit of matter (for example a crystal or an ideal gas), the macrostate is completely independent of the observer: it’s just the state in which the law of physics say that bit of matter should be. In an ideal gas it is entirely determined by the pressure and volume, which are anything but subjective. For a crystal it is more complex because we have to account for things like its shape but the reasoning is the same.

Then, the microstates are just accessible states, and this is also dictated by Physics. For example, it is quite easy to see that a crystal has fewer accessible states than a gas (the atoms’ positions are constrained and the velocities are limited to the crystal’s vibration modes). We can calculate the entropy in the experimental conditions within that framework, or in the case of correlated liquids, or amorphous solids, or whatever. But the fact that we can come up with different entropies if we make different hypotheses does not mean that any of these hypotheses is actually valid. If we measure the entropy directly we might have a value that is consistent with several models, or none. The actual entropy is what we observe, not the theoretical scaffolding we use to try to make sense of it. And again, this is not subjective.

Re: What is entropy? A measure of just how little we know

#95

Post by kergonath »

kgwgk wrote: post #93
kergonath wrote: post #92

Earlier quoted context omitted.

Plenty of bad stuff made its way into textbooks, I saw some really dodgy stuff at uni. And for every uncontroversial statement we can find a textbook that argues that it is wrong. Sorting the good from the bad is the main point of studying science and it is not easy. What is also important is that approaches that can work in some field or context might be misleading or lead to wrong outcomes in others. Information th…

Plenty of bad stuff makes its way into hackernews comments as well. Saying that “it could be wrong” doesn’t really support the “it’s wrong” claim, does it?

I did not make that point, though. I rejected an appeal to authority because something was in a textbook. I made no comment about the validity of that person’s work in his field, I just pointed out that this transferability was limited.

Re: What is entropy? A measure of just how little we know

#96

Post by kgwgk »

kergonath wrote: post #95
kgwgk wrote: post #93

Earlier quoted context omitted.

Plenty of bad stuff makes its way into hackernews comments as well. Saying that “it could be wrong” doesn’t really support the “it’s wrong” claim, does it?

I did not make that point, though. I rejected an appeal to authority because something was in a textbook. I made no comment about the validity of that person’s work in his field, I just pointed out that this transferability was limited.

My bad, I thought you considered Balian’s work another misguided attempt to apply information theory concepts to thermodynamics.

For the record, this is the abstract of the “Information in statistical physics” article: “We review with a tutorial scope the information theory foundations of quantum statistical physics. Only a small proportion of the variables that characterize a system at the microscopic scale can be controlled, for both practical and theoretical reasons, and a probabilistic description involving the observers is required. The criterion of maximum von Neumann entropy is then used for making reasonable inferences. It means that no spurious information is introduced besides the known data. Its outcomes can be given a direct justification based on the principle of indifference of Laplace. We introduce the concept of relevant entropy associated with some set of relevant variables; it characterizes the information that is missing at the microscopic level when only these variables are known. For equilibrium problems, the relevant variables are the conserved ones, and the Second Law is recovered as a second step of the inference process. For non-equilibrium problems, the increase of the relevant entropy expresses an irretrievable loss of information from the relevant variables towards the irrelevant ones. Two examples illustrate the flexibility of the choice of relevant variables and the multiplicity of the associated entropies: the thermodynamic entropy (satisfying the Clausius–Duhem inequality) and the Boltzmann entropy (satisfying the H-theorem). The identification of entropy with missing information is also supported by the paradox of Maxwell’s demon. Spin-echo experiments show that irreversibility itself is not an absolute concept: use of hidden information may overcome the arrow of time.”

Re: What is entropy? A measure of just how little we know

#97

Post by kayo_20211030 »

Why has Quanta degenerated into a mad combination of ineffective science communication and breathless, and slightly envious references to yoga retreats in the North of England. It's become a lifestyle magazine for the self-important, science-adjacent middlebrows. Most obviously, it's nerd-sniping for HN.

And before you _instinctively_ (it will be instinctive) downvote me to oblivion, please read the piece and assure yourself that it's not truly a piece of rote trash.

Re: What is entropy? A measure of just how little we know

#98

Post by oh_my_goodness »

kergonath wrote: post #94
oh_my_goodness wrote: post #71

Earlier quoted context omitted.

"We need to be careful", agreed. "It's not mundane", agreed. (It's mundane in information theory because that's how they define entropy.) "It's [...] not right" (from your first comment), can you give/link a specific physical example? It would be very cool to have a clear counterexample.

> can you give/link a specific physical example? About the lack of subjectivity of the states? If we consider any bit of matter (for example a crystal or an ideal gas), the macrostate is completely independent of the observer: it’s just the state in which the law of physics say that bit of matter should be. In an ideal gas it is entirely determined by the pressure and volume, which are anything but subjective. For a…

Agreed, sure. Of course it's not subjective.

Is there a concrete physical example where the information-theory definition of entropy conflicts with experiment?

Re: What is entropy? A measure of just how little we know

#100

Post by whatshisface »

kergonath wrote: post #90
whatshisface wrote: post #79

Earlier quoted context omitted.

The enthalpy changes measured by a calorimeter are dependent on the design of the calorimeter, which could have been a different piece of equipment. In a sense, that makes it dependent on the definition of enthalpy. If you introduced a new bit of macro information to the definition of an ensemble, you'd divide the number of microstates by some factor. That's the micro level equivalent of macroscopic entropy being und…

> The enthalpy changes measured by a calorimeter are dependent on the design of the calorimeter, which could have been a different piece of equipment. Right, but that is true of anything. Measuring devices need to be calibrated and maintained properly. It does not make something like a distance subjective, just because someone is measuring it in cm and someone else in km. > If you introduced a new bit of macro inform…

Entropy isn't subjective, but it is relative to what you can measure about a given system, or what measurements you are analyzing a system with respect to. In quantum mechanics there are degenerate ground states, and in classical mechanics there are cases where the individual objects (such as stars) are visible but averaged over. You should take a look at the Gibbs paradox.
Post reply on HN