Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It took me 10 years to understand entropy

cantorsparadise.com

Post reply on HN
Print view View original

151–160 of 295 posts

Re: It took me 10 years to understand entropy

#151

Post by oh_my_goodness »

kgwgk wrote: post #145
oh_my_goodness wrote: post #144

Earlier quoted context omitted.

Sure. A fine-grained macro-state contains fewer micro-states. A coarse-grained macro-state contains more micro-states. Say I flip 8 coins and I don't look at the results. A fine-grained macro state is TTTT TTTT. A coarser-grained macro state is TTTT xxxx. The one has 4 bits more entropy than the other. It works the same way in statistical mechanics. Call them spins. We're just talking about some ensemble of micro sta…

How is a macrostate TTTTxxxx different from having the information about the TTTT part and not about the xxxx part? Talking bout the entropy of the macrostate TTTTxxxx makes sense only conditional on the TTTT information.

It's different because I can define a macro-state without any information about which macro-state any system is actually in. As I think you're also saying, the only information I need is information about how I've defined my own macro-states.

If we just define the macro-states, we're good to go. We don't need to talk about 'knowledge'. We can talk about 'knowledge', it's fine, but that lets in unnecessary woo.

Re: It took me 10 years to understand entropy

#152

Post by IX-103 »

kgwgk wrote: post #123
oldsecondhand wrote: post #121

Earlier quoted context omitted.

You can observe the movement of molecules beyond macro properties like temperature.

Sure, at least in principle. And if you knew what every molecule was doing the entropy would vanish.

I've thought that for a while, but I'm not a physicist. Do you know any prominent physicists that hold that view? It seems to contradict at least the popular narrative about entropy as a property of a system.

Re: It took me 10 years to understand entropy

#153

Post by jbay808 »

oh_my_goodness wrote: post #144
jbay808 wrote: post #143

Earlier quoted context omitted.

Can you elaborate on the difference between a "fine-grained" macrostate and a macrostate that is not fine-grained? I think you will find it hard to separate the concept of a macrostate from the state of knowledge (or ignorance) of an individual subjective observer.

Sure. A fine-grained macro-state contains fewer micro-states. A coarse-grained macro-state contains more micro-states. Say I flip 8 coins and I don't look at the results. A fine-grained macro state is TTTT TTTT. A coarser-grained macro state is TTTT xxxx. The one has 4 bits more entropy than the other. It works the same way in statistical mechanics. Call them spins. We're just talking about some ensemble of micro sta…

> A fine-grained macro state is TTTT TTTT.

This macrostate seems fine enough to be a microstate, but sure. For the macrostates with at least one 'x' in it, that 'x' seems to be a placeholder for the concept of subjective ignorance.

> That doesn't mean I necessarily have any information about which macro-state the system is actually in.

But the entire purpose of the exercise of assigning microstates to macrostates is so that you can match up a description of some system to a microstate ensemble and calculate its entropy! Otherwise there's no point to arbitrarily labelling various groups of microstates.

To follow your example more practically, let's say you have an 8-spin system, whose net spin is zero. (You know because you've measured its overall magnetic moment or something). I've just described a system that is in one of the following possible microstates:

TTTT HHHH, TTTH HHHT, ..., HHHH TTTT

Now you can go ahead and define the macrostates as fine-grained as you want, where TTTT HHHH and HHHH TTTT are in different macrostates, but to calculate the entropy of this system, you're going to have to sum up all of those macrostates anyway to get the one that's consistent with the described system.

Re: It took me 10 years to understand entropy

#154

Post by kgwgk »

oh_my_goodness wrote: post #151
kgwgk wrote: post #145

Earlier quoted context omitted.

How is a macrostate TTTTxxxx different from having the information about the TTTT part and not about the xxxx part? Talking bout the entropy of the macrostate TTTTxxxx makes sense only conditional on the TTTT information.

It's different because I can define a macro-state without any information about which macro-state any system is actually in. As I think you're also saying, the only information I need is information about how I've defined my own macro-states. If we just define the macro-states, we're good to go. We don't need to talk about 'knowledge'. We can talk about 'knowledge', it's fine, but that lets in unnecessary woo.

I'm not sure I see what's the point of that distinction.

The entropy of a macrostate is a measure of the indetermination about the microstate conditional on the macrostate. If you don't want to call that 'knowledge' the substance of the matter doesn't change.

A macrostate is not an intrinsict property of a physical system. It's related to our description of the system. In general, the same microstate of the system of interest may be compatible with multiple macrostates.

Given the thermodynamical description of some system I can calculate the entropy if T=T_1 and the entropy if T=T_2 without knowing what's the actual temperature specifying the macrostate. But in the first case the calculation is conditional on the hypothetical information T=T_1 and in the second case conditional on T=T_2.

Re: It took me 10 years to understand entropy

#155

Post by oh_my_goodness »

jbay808 wrote: post #153
oh_my_goodness wrote: post #144

Earlier quoted context omitted.

Sure. A fine-grained macro-state contains fewer micro-states. A coarse-grained macro-state contains more micro-states. Say I flip 8 coins and I don't look at the results. A fine-grained macro state is TTTT TTTT. A coarser-grained macro state is TTTT xxxx. The one has 4 bits more entropy than the other. It works the same way in statistical mechanics. Call them spins. We're just talking about some ensemble of micro sta…

> A fine-grained macro state is TTTT TTTT. This macrostate seems fine enough to be a microstate, but sure. For the macrostates with at least one 'x' in it, that 'x' seems to be a placeholder for the concept of subjective ignorance. > That doesn't mean I necessarily have any information about which macro-state the system is actually in. But the entire purpose of the exercise of assigning microstates to macrostates is…

Good review of common ground. At this point hopefully the active folks in the discussion can see that we're all describing statistical mechanics exactly the same way.

What I'm saying is pedagogical. We need to define our macro-states. We don't need to go on and talk about our definitions being information or 'knowledge.' We could just use the definitions and calculate. We can talk about 'knowledge' but we don't need to.

The exception is when we actually have some information about what macro-state some system is really in. Obviously we then have to build that information into our model, and the entropy changes. What I'm saying is this: it's not necessary to mix that into our definition of entropy. That definition is not going to help folks who don't understand entropy, and it's unnecessary.

Re: It took me 10 years to understand entropy

#156

Post by im3w1l »

kgwgk wrote: post #154
oh_my_goodness wrote: post #151

Earlier quoted context omitted.

It's different because I can define a macro-state without any information about which macro-state any system is actually in. As I think you're also saying, the only information I need is information about how I've defined my own macro-states. If we just define the macro-states, we're good to go. We don't need to talk about 'knowledge'. We can talk about 'knowledge', it's fine, but that lets in unnecessary woo.

I'm not sure I see what's the point of that distinction. The entropy of a macrostate is a measure of the indetermination about the microstate conditional on the macrostate. If you don't want to call that 'knowledge' the substance of the matter doesn't change. A macrostate is not an intrinsict property of a physical system. It's related to our description of the system. In general, the same microstate of the system of…

I think in practice we select macrostates based on stuff we can easily measure. And stuff we can easily measure gets pretty close to intrinsic properties.

Re: It took me 10 years to understand entropy

#157

Post by denton-scratch »

mhh__ wrote: post #88
l33t2328 wrote: post #48

Earlier quoted context omitted.

Not to poke holes in your nostalgia, but how do you know he had a great explanation if you don’t remember it after further study?

"Don't try to understand it, feel it" - from tenet, but does sort of apply to entropy as a way of looking at problems. That being said I heard a sports science student try to recall their working definition of entry and it was some mess of locks and keys floating around randomly hitting eachother?

> definition of entry and it was some mess of locks and keys floating around randomly hitting eachother?

Locks and keys does indeed sound like "entry". I thought we were discussing "entropy".

Re: It took me 10 years to understand entropy

#158

Post by lazide »

RappingBoomer wrote: post #66
lazide wrote: post #10

Earlier quoted context omitted.

Does it hurt to try?

it hurts to make decisions based on our current knowledge that is a child's knowledge

How exactly? And how exactly would we ever get past a ‘child’s knowledge’ without learning and making mistakes?

Re: It took me 10 years to understand entropy

#159

Post by jwarden »

andi999 wrote: post #81
jbay808 wrote: post #73

Earlier quoted context omitted.

> But if you look deeper than that averaging it stops making sense to me. It's a completely different world. I think you're less confused than you think you are! As I posted elsewhere, it helps to think of entropy as a quantity that actually depends on how much you know about the system in question. Typically when you calculate the entropy of a system at temperature X, that means all you know is that you stuck a ther…

Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which parameters you hold fixed.

Fascinating discussion. I see some parallel here to the Bayesian vs Frequentist view of probability.

They are perhaps both valid points of view depending on the situation.

If you take a frequentist view of an unbiased coin, then the probability that it will land heads on the next flip is objectively, by definition 50%. So the resulting calculation of entropy (log 50% = 1 bit) is also objectively defined. But if your 50% probability represents a subjective belief, the resulting entropy calculation should also be considered subjective, I would think.

Re: It took me 10 years to understand entropy

#160

Post by jbay808 »

oh_my_goodness wrote: post #142
toxik wrote: post #114

Earlier quoted context omitted.

Leonard Susskind disagrees with you. See his lectures on statistical mechanics, he is very clear that entropy is a matter of knowledge about the system. It has to be.

Susskind says that entropy is determined by selecting a macro-state. He doesn't claim that the entropy of a macro-state depends on whether we know which macro-state the real system is really in. If we happen to know, then, sure. For example we could pick a weird-ass observable state, and when we saw it we would know the entropy of the system was low. But the entropy of each macro-state just depends on how many micro-…

The concept of entropy wasn't invented so that we could calculate entropies of macrostates, it was so that we could calculate entropies of real systems and understand their behaviour. Macrostates are an accounting tool that helps us do this. You seem to be treating the calculation of macrostate entropy as an end-goal in itself, but also allowing yourself to somehow freely choose any macrostate you want. When it comes to applying thermodynamics in practice, you'll have to calculate the entropy of a real, or at least hypothetical, system.

The point of macrostates is that you ought to know which macrostate a given system is in. That's the thing that you know. You don't know which microstate it's in, but you do know which macrostate it's in.

For example, if I say "a cubic metre volume of air at room temperature and pressure", I've described a physical system. I've also described a macrostate.

If you're calculating the entropy of macrostates that are not consistent with a description of a system -- if you've defined your macrostates such that you don't know which macrostate a given system is in -- then in order to calculate that system's entropy you have to sum up over all such possible macrostates anyway, so you haven't saved yourself any work or earned any insights along the way.

So yes, you can calculate the entropy of a macrostate without knowing what macrostate a real system is in, but it kind of sounds like you're arguing that log(x) is not a function of variable x, because log(3) is a constant and log(4) is a constant, and you can divide up any x into constants of your choice.

Post reply on HN