Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is entropy? A measure of just how little we know

quantamagazine.org

Post reply on HN
Print view View original

151–160 of 169 posts

Re: What is entropy? A measure of just how little we know

#151

Post by tsimionescu »

kgwgk wrote: post #147
tsimionescu wrote: post #145

Earlier quoted context omitted.

The post I was replying to claims that probability is not a physical property of an observed system, that it is a property of an observer trying to observe a system. The examples given in the quoted link all talk about experiments like rolling dice or tossing coins, and explain that knowledge of mechanics shows that these things are perfectly predictable mechanical processes, and so any "probability" we assign to the…

> an observer may have much less information than physically possible, but it can't have more information about some system than a physical limit. Still the probability represents the uncertainty of the observer. You say that "the most possible information" is still not enough because "measurement is a time-consuming process" and it's not "possible to measure" with infinite precision. I'd say that you're just confirm…

I would say that any state that we can't measure is not a physical state. So if we can only measure, even in theory, a system up to precision dx, then the system being in state a means that some quantity is x±dx. If the rules say that the system evolves from states where y>=x to state A, and from states where yThis is similar to quantum measurement: when we see that the particle was here and not there, we don't learn anything new about its quantum state before the measurement. We already knew everything there was to know about the particle, but that didn't help us predict more than the probabilities of where it might be.

Re: What is entropy? A measure of just how little we know

#152

Post by BenoitEssiambre »

tsimionescu wrote: post #119
BenoitEssiambre wrote: post #117

Earlier quoted context omitted.

It is though. Temperature is an aggregate summary statistic used when the observer doesn't know the details of individual particles. If you did know their position, speed and velocities, you could violate the laws of entropy as Maxwell's thought experiment demonstrated in 1867 https://en.wikipedia.org/wiki/Maxwell%27s_demon

Maxwell's demon is a thought experiment that involves a magical being. It's an interesting thought experiment, and it provides some insights about the relationship between macro states and micro states, but it's not actually a refutation of anything, and it doesn't describe any physically realizable system, even in theory. There is no way to build a physical system that can act as the demon, and so it follows that en…

"large enough" and "bath" is doing a lot of work here. I don't think it's necessarily about size. It's about that size and configuration implying the particle states are difficult to know.

If for example, you had a large size but the states were knowable because they were all correlated, they were following a functionally predictable path, for example all moving away from the ice cube, or all orderly orbiting around the ice cube in a centrifuge such that they didn't quite touch the ice cube, it wouldn't melt.

Re: What is entropy? A measure of just how little we know

#153

Post by TexanFeller »

swyx wrote: post #129
TexanFeller wrote: post #88

Entropy got a lot more exciting to me after hearing Sean Carroll talk about it. He has a foundational/philosophical bent and likes to point out that there are competing definitions of entropy set on different philosophical foundations, one of them seemingly observer dependent: - https://youtu.be/x9COqqqsFtc?si=cQkfV5IpLC039Cl5 - https://youtu.be/XJ14ZO-e9NY?si=xi8idD5JmQbT5zxN Leonard Susskind has lots of great talks…

why is it exciting at all? it is the most depressing concept i have ever heard.

The exciting part is there are different concepts of it that start from different foundations and arrive at the same thing. It’s suggestive that our understanding is incomplete and there are deeper levels of understanding to uncover. And it _could_ just be the case that the universe started with low entropy for no reason, but if there are reasons and we ponder them it could lead to some massive epiphanies in physics/cosmology.

Re: What is entropy? A measure of just how little we know

#154

Post by kgwgk »

tsimionescu wrote: post #151
kgwgk wrote: post #147

Earlier quoted context omitted.

> an observer may have much less information than physically possible, but it can't have more information about some system than a physical limit. Still the probability represents the uncertainty of the observer. You say that "the most possible information" is still not enough because "measurement is a time-consuming process" and it's not "possible to measure" with infinite precision. I'd say that you're just confirm…

I would say that any state that we can't measure is not a physical state. So if we can only measure, even in theory, a system up to precision dx, then the system being in state a means that some quantity is x±dx. If the rules say that the system evolves from states where y>=x to state A, and from states where y This is similar to quantum measurement: when we see that the particle was here and not there, we don't lear…

> I would say that any state that we can't measure is not a physical state.

Ok, so if I understand correctly for you microstates are not physical states and it doesn't make sense to even consider that at any given moment the system may be in a particular microstate. That's one way to look at things but the usual starting point for statistical mechanics is quite different.

Re: What is entropy? A measure of just how little we know

#155

Post by jedharris »

plank wrote: post #122
mike_ivanov wrote: post #85

Earlier quoted context omitted.

Interesting. Could you share a link to your thesis?

Sure. Had to do some searching;-) Info on thesis: https://dare.uva.nl/search?identifier=0ae63403-264b-4bf0-91c... The document itself (self hosted) https://gofile.me/7uDSJ/sGJCFD3W7 Probably most important article: (sorry, only abstract): https://journals.aps.org/pra/abstract/10.1103/PhysRevA.54.24...

PDF available at https://dare.uva.nl/search?identifier=41ed9d6d-70a7-4bc7-882...>

Re: What is entropy? A measure of just how little we know

#156

Post by kgwgk »

tsimionescu wrote: post #149
kgwgk wrote: post #146

Earlier quoted context omitted.

> the entropy of a thermodynamic system is proportional to the amount of microstates (positions, types, momentum, etc. of individual particles) that would lead to the same macrostate (temperature, volume, pressure, etc.). It doesn't matter if an observer is aware of any of this, it's an objective property of the system. The meaning of "would lead to the same macrostate" (and therefore the entropy) is not an "objectiv…

> That's what people mean when they say that the entropy is not an "objective" property of a physical system: that it depends on how we choose to describe that physical system (and that description is external to the physical system itself). I understand that's what they mean, but this is the part that I think is either trivial or wrong. That is, depending on your choice you'll of course get different values, but it…

> I understand that's what they mean, but this is the part that I think is either trivial or wrong. That is, depending on your choice you'll of course get different values, but it won't change anything about the system. It's basically like choosing to measure speed in meters per second or in furlongs per fortnight, or choosing the coordinate system and reference frame: you get radically different values, but relative results are always the same.

I would agree that it's trivial but then it's equally trivial that it's not just like a change of coordinates.

Say that you choose to represent the macrostate of a volume of gas using either (a) its pressure or (b) the partial pressures of the helium and argon that make it up. If you put together two volumes of the same mixture the entropy won't change. The entropy after they mix is just the sum of the entropies before mixing.

However when you put together one volume of helium and a one volume of argon the entropy calculated under choice (a) doesn't change but the entropy calculated under choice (b) does increase. We're not calculating the same thing in different units: we're calculating different things. There is no change of units that makes a quantity change and also remain constant!

The (a)-entropy and the (b)-entropy are different things. Of course it's the same concept applied to two different situations but that doesn't mean it's the same thing. (Otherwise one could also say that the momentum of a particule doesn't depend on its mass or velocity because it's always the same concept applied in different situations.)

Re: What is entropy? A measure of just how little we know

#157

Post by ljosifov »

I didn't see this Sabine video mentioned in the comments. Maybe of interest to someone interested in entropy in physics.

"I don't believe the 2nd law of thermodynamics. (The most uplifting video I'll ever make.)"

https://m.youtube.com/watch?v=89Mq6gmPo0s

I come to entropy from Machine Learning, Information Theory and probability. For me it's fairly straightforwad. Of interest and useful - but nothing mysterious there.

The p.d.f. aka a fancy histogram is my current best knowledge of how many times an outcome is expected to happen. When I do one experiment - I can't tell the outcome. I can only count the number of different outcomes, without being certain when will any of them appear exactly.

A flat p.d.f. means my knowledge is poor: every outcome is about equaly possible. I'm very ignorant (high entropy). A spiky p.d.f. means I have good knowledge: some outcome is much more likely (low entropy). In extremis the p.d.f. is a Dirac impulse - that's deterministic knowledge.

The only mildly interesting thing is when a new observation reduces my knowledge. Say right now I'm fairly certain I have not got cancer. My chances are 90:10 for my age. Tomorrow I take a test, the test comes back positive. Of people of my age that test positive, aboout half have cancer for real. After the test my chances are 50:50. Now I am perfectly ignorant whether I have cancer or not. Whereas before I took the test and got a positive result, I was very certain I have not got cancer. The new information (positive test result), transformed my probability of cancer from spiky marginal P_Y(y)={0.9,0.1} to perfectly ignorant conditional P_Y(y|+ve test)={0.5,0.5}.

This example is from "How to measure the information gained from one symbol" by DeWeese and Meister (https://pubmed.ncbi.nlm.nih.gov/10695762/).

Re: What is entropy? A measure of just how little we know

#158

Post by Jensson »

BenoitEssiambre wrote: post #152
tsimionescu wrote: post #119

Earlier quoted context omitted.

Maxwell's demon is a thought experiment that involves a magical being. It's an interesting thought experiment, and it provides some insights about the relationship between macro states and micro states, but it's not actually a refutation of anything, and it doesn't describe any physically realizable system, even in theory. There is no way to build a physical system that can act as the demon, and so it follows that en…

"large enough" and "bath" is doing a lot of work here. I don't think it's necessarily about size. It's about that size and configuration implying the particle states are difficult to know. If for example, you had a large size but the states were knowable because they were all correlated, they were following a functionally predictable path, for example all moving away from the ice cube, or all orderly orbiting around…

The ice cube would melt, it would just take longer, heat does transfer through vacuum via radiation. And no, you can't control the positions of the electrons to stop that radiation from happening, it is a quantum effect not something you can control.

Re: What is entropy? A measure of just how little we know

#160

Post by tsimionescu »

kgwgk wrote: post #156
tsimionescu wrote: post #149

Earlier quoted context omitted.

> That's what people mean when they say that the entropy is not an "objective" property of a physical system: that it depends on how we choose to describe that physical system (and that description is external to the physical system itself). I understand that's what they mean, but this is the part that I think is either trivial or wrong. That is, depending on your choice you'll of course get different values, but it…

> I understand that's what they mean, but this is the part that I think is either trivial or wrong. That is, depending on your choice you'll of course get different values, but it won't change anything about the system. It's basically like choosing to measure speed in meters per second or in furlongs per fortnight, or choosing the coordinate system and reference frame: you get radically different values, but relative…

> However when you put together one volume of helium and a one volume of argon the entropy calculated under choice (a) doesn't change but the entropy calculated under choice (b) does increase. We're not calculating the same thing in different units: we're calculating different things. There is no change of units that makes a quantity change and also remain constant!

Agreed, this is not like a coordinate transform at all. But the difference from a coordinate transform is that they are not both equally valid choices for describing the physical phenomenon. Choice (a) is simply wrong: it will not accurately predict how certain experiments with the combined gas will behave.

Post reply on HN