Live data from Hacker News

What is entropy? A measure of just how little we know

quantamagazine.org

31–40 of 169 posts

Re: What is entropy? A measure of just how little we know

#31
post #2

> As physicists have worked to unite seemingly disparate fields over the past century, they have cast entropy in a new light — turning the microscope back on the seer and shifting the notion of disorder to one of ignorance. Entropy is seen not as a property intrinsic to a system but as one that’s relative to an observer who interacts with that system. Maybe I have the benefit of giant shoulders, but this seems like a…

> Maybe I have the benefit of giant shoulders, but this seems like a fairly mundane observation.

It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics.

> High-entropy states are those macrostates which have many corresponding microstates.

That is how you deduce entropy form a given model. But entropy is also something that we can get from experimental measurements. In this case, the experimental setup does not care about microstates and macrostates, it just has properties like enthalpy, heat capacity and temperature.

We can build models after the fact and say that e.g. the entropy of a given gas matches that predicted by our model for ideal gases, or that the entropy of a given solid matches what we know about vibrational entropy.

That’s how we say that e.g. hydrogen atoms are indistinguishable. It’s not that they become indistinguishable because we decide so. It’s because we can calculate entropy in both cases and reality does not match the model with distinguishable atoms.

> The classification of several microstates into the same macrostate, is this not a distinctly observer-centred function?

It seems that way if we consider only our neat models, but it fails to explain why experimental measurements of the entropy of a given materials are consistent and independent of whatever model the people doing the experiment were operating on. Fundamentally, entropy depends on the probability distribution, not the observer.

Re: What is entropy? A measure of just how little we know

#32
post #30

Earlier quoted context omitted.

What anthropic do you mean? I'm describing properties of models, not people. Physics (probably) doesn't care what you "know". On the guid idea - fundamental particles are indistinguishable from one another in quantum mechanics, so they don't have anything like a guid even in principle. There is no experiment you could perform on an electron to determine whether it had been swapped out for a "different" one, for insta…

Sorry ... I am replying mostly to the dice idea which wasn't you. Yes correct about the guid idea. My point is the discussion is easier to follow if grounded in reality (as best modelled since that is all we have plus some evidence stored in the same "SSD"!)

Oh I see. But on your guid thing, people often describe entropy in terms of the set of micro states of your system (the actually physical states in your model) and the macro states (sets of microstates that are described by a collection of high-level state variables like pressure/temperature).

Physically indistinguishable stuff would have the same micro state, so yeah, they wouldn't affect entropy calculations at all, no matter what macro states you picked.

But I disagree a bit about grounding things in reality - some concepts are quite abstract and having clean examples can be helpful, before you start applying them to the mess that is our universe!

Re: What is entropy? A measure of just how little we know

#33

Earlier quoted context omitted.

In physics the requirement for a valid changing the frame of reference is, that the laws of physics transform according to the transformation. Every observer should discover the same fundamental laws when performing experiments and using the scientific method. To stay in your analogy, saying 5 and 6 are the same would only work if the rules of the game you play could transform in such a way that an observer making a…

I think you may have misunderstood the OP's point - the entropy you calculate for a system depends on how you factor the system into micro and macro states. This doesn't really have anything to do with changes of reference frame - in practice it's more about limitations on the kinds of measurements you can make of the system. (you can't measure the individual states of all the particles in a body of gas, for instance…

From a thermodynamics point of view only the differential of the entropy matters, so if there is only a fixed difference between the two computations they do not influence the physics.

If the way one does the coarse graining of states results in different differentials, one way should be the correct one.

There is only one physics.

If I remember one of Plancks relevations was that he could explain why a certain corrections factor was needed in entropy calculations, since phase space had finished cell size.

Re: What is entropy? A measure of just how little we know

#34

Lovely article. The subjective nature of entropy and information immediately makes me think of the IIT theory (integrated information theory) of consciousness and its foundational futility. Information cannot be discussed without perspective. Someone has to define the states. A die have 6 states only to us humans. What about an ant that's likely to have the die land on it? Bringing the observer back into discussions…

For me CS with Physics BSc, it's always fascinating to still see how entropy is still sich a wild measure.

Re: What is entropy? A measure of just how little we know

#36
post #35

Makes sense. If we knew the theory of everything and the initial conditions of the universe, then we could just compute the next episode of a series instead of streaming it.

Not necessarily. A theory of everything does not imply either the computability of a future state or determinism.

Even if our ToE is deterministic the universe may be computationally irreducible, meaning it cannot be computed accurately at lower resolution in all cases. Note that such a universe could contain within it regions that are computationally reducible, just not the whole and not all regions.

I would expect a ToE to give us knowable bounds to either determinism or computability. It should tell us what is precisely knowable or predictable and what isn’t.

Edit: to understand how a ToE could leave some things unknowable (but tell us what they are) consider the Hubble horizon. Light beyond it will never reach us making sufficiently distant things unknowable.

Limits may be great. It means we can at least subjectively consider ourselves as having free will — even with a deterministic theory it may be unknowable determinism. It’s just like how the speed of light might be why we got to evolve before being bum rushed by aliens.

Re: What is entropy? A measure of just how little we know

#37
I remember that somewhere during my physics degree this idea of the article clicked with me.

Entropy is just a simpler way of manifesting properties of a system that we didn’t measure because it is too difficult (when I was still at classical physics level) or it’s not possible to measure (when quantum mechanics became present in everything we learned).

We use it because we can’t measure the position and momentum of every particle (same goes for temperature) so we created theories on how a system with a particular entropy behaves, but it’s just a clever way to work with aproximations.

I find this idea fascinating.

Re: What is entropy? A measure of just how little we know

#38
post #2

> As physicists have worked to unite seemingly disparate fields over the past century, they have cast entropy in a new light — turning the microscope back on the seer and shifting the notion of disorder to one of ignorance. Entropy is seen not as a property intrinsic to a system but as one that’s relative to an observer who interacts with that system. Maybe I have the benefit of giant shoulders, but this seems like a…

> mundane observation What makes an observation mundane? I think what you said is insightful and demonstrates intelligence. I don't think it's at all obvious to the masses of students that have poured over introductory physics textbooks. In fact, it seems to me that often entropy is taught poorly and that very few people understood it well but that we are beginning to correct that. I point to the heaps of popsci maga…

Maybe if you opened only introductory physics textbooks then it’s not mundane; but If you opened introductory information theory textbooks, statistical textbooks, introductions to Bayesian probability theory, articles about MaxEnt thermodynamics including articles by E.T. Jaynes, then it’s a quite mundane observation.

Re: What is entropy? A measure of just how little we know

#39
post #26

Interesting to read this, 27 years after my PhD* (theoretical physics), in which I did compare the view WITH and the view WITHOUT ‘unknowns’ causing entropy as a driver. * My PhD was about how to treat a (quantum mechanical) system inside a cavity: a cavity with one perfect mirror and one 99.999999% perfect mirror. The (one dimensional) universe was made whole by another perfect mirror at the other side of the non-pe…

I read you're comment with interest, but ultimately I can't understand the point being made because I don't know what kind of mirror you're referring to (optical?), I don't know what 'l' or 'L' represent (lateral spacing of mirrors?, vacuum energy desnities?), and the last sentence I think maybe the word 'how' should be deleted?

Re: What is entropy? A measure of just how little we know

#40
post #2

> As physicists have worked to unite seemingly disparate fields over the past century, they have cast entropy in a new light — turning the microscope back on the seer and shifting the notion of disorder to one of ignorance. Entropy is seen not as a property intrinsic to a system but as one that’s relative to an observer who interacts with that system. Maybe I have the benefit of giant shoulders, but this seems like a…

> Maybe I have the benefit of giant shoulders, but this seems like a fairly mundane observation. It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. > High-entropy states are those macrostates which have many corresponding microstates. That is how you deduce entropy form a given model. But entropy is also something that we can get from experimental measurements. In this ca…

> It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics.

Articles about MaxEnt thermodynamics by E.T. Jaynes where he talks about the “anthropomorphic” nature of entropy date back to 1960s. How is that not right in physics?

Post reply on HN