Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It took me 10 years to understand entropy

cantorsparadise.com

Post reply on HN
Print view View original

201–210 of 295 posts

Re: It took me 10 years to understand entropy

#201

Post by [deleted] »

jbay808 wrote: post #170
im3w1l wrote: post #149

Earlier quoted context omitted.

I've been trying to reconcile these perspectives, and I think it really is both. And they are both physically relevant. Consider the subjective entropy perspective. If you know the exact microstate of a system, then you can in theory play the part of Maxwell's demon. You could have a little gate that you open only for fast particles, and using your knowledge of the microstate, you can predict exactly when they will a…

> If you take this very same system and put it in thermal contact with another system, then an objective entropy perspective is the relevant one. Those systems will equilibrize and your subjective knowledge is irrelevant to that process. The subjective view handles this scenario just fine, though, and makes more accurate predictions than the objective view. For example, there are systems where some aspects of the ori…

[deleted]

Re: It took me 10 years to understand entropy

#202

Post by jbay808 »

coldtea wrote: post #185
jbay808 wrote: post #183

Earlier quoted context omitted.

"similar" is in the eye of the observer! This fact should be especially clear in the context of bit strings. If 10010001111110101000 is my login password, don't be surprised if other permutations fail to grant you access, even if you have the correct number of 1's and 0's.

> "similar" is in the eye of the observer! Nope, also similar in algorithmic complexity theory (e.g. counting compressibility).

That's a very good, and deep, point, and I agree that there is something important there. However, algorithmic complexity is still only defined relative to an arbitrary reference computer. If my reference computer happens to, in hardware, XOR its inputs and outputs with the bitstring 10010001111110101000, then (in terms of bits that end up represented on the drive), that will be the one that has the lowest algorithmic complexity, although the algorithm might think that it's outputting all 0's.

Re: It took me 10 years to understand entropy

#203

Post by Jensson »

kgwgk wrote: post #199
Jensson wrote: post #198

Earlier quoted context omitted.

Your statement doesn't make sense, temperature is defined in terms of entropy changes, you can't calculate temperature without first calculating entropy changes.

Have you heard of thermometers? I can have a container with 1l of some gas at room temperature T1 and proceed to heat the room - and the container - to temperature T2. How do you calculate the number of microstates for the sample of gas before and after? How do you think these numbers are related? You said it was easy!

Thermometers is a way to measure temperature, it isn't its theoretical definition. Temperature is defined as energy required per change in entropy. There is no other reasonable way to define temperature, since at its core it measures which way energy flows when two macro systems are connected. Temperature tends to go up as you add energy to things, but not always, for example temperature doesn't go up when you melt ice, it starts and ends at 0 degrees C.

> How do you calculate the number of microstates for the sample of gas before and after? How do you think these numbers are related?

If you added energy to the gas by heating it, lets say you doubled the energy, then you now have twice as many energy packets to distribute between the particles. This adds a lot more microstates that wasn't available before, and none of the old microstates are now possible since all old microstates had a total energy level half of what each new microstate has. You can calculate the change in states yourself, it is just discrete normal probability theory. Note that the base rate isn't interesting, you care about the change of the logarithm of number of states.

Re: It took me 10 years to understand entropy

#204

Post by andi999 »

jbay808 wrote: post #187
oh_my_goodness wrote: post #168

Earlier quoted context omitted.

We seem to be stuck in a loop of explaining basic first-year statistical mechanics back and forth to each other repeatedly. I'm not sure why. I'm making a pedagogical point. The OP addresses how difficult entropy is to understand. I'm responding to that. We don't need to talk about "knowledge" when you define entropy, or in an initial explanation of entropy. We could, but we could decide not to. The log(x) example is…

Alright. I agree we seem to be stuck in a weird loop where we agree about all the observable facts but somehow are on different wavelengths in spite of that. And I totally agree, entropy is a property of a macrostate. The information step comes inseparably in when you go from a system description to a macrostate. And you might just shuffle the confusion from not knowing what entropy is to not knowing what a macrostat…

Why shd the entropy of a brick not be an objective property (apart from a constant). I mean you can measure it, isnt it basically the integral of C/T dt?

Re: It took me 10 years to understand entropy

#206

Post by zwieback »

Of course you don't need to really understand entropy for it to be useful. It's definitely an interesting concept but when I was crunching equations for Thermodynamics, one of the weeder classes for ME, it becomes clear you need it for things to balance out. Once you've cranked threw a dozen or so problems you get a feel for what it is even if the physics and the spiritual side of it remains murky.

Now, 35 years later, when I marvel at my new engine or what have you, I still vaguely remember my entropy-problems days and appreciate that someone worked this stuff out.

Re: It took me 10 years to understand entropy

#207

Post by mannykannot »

alecst wrote: post #178
andi999 wrote: post #81

Earlier quoted context omitted.

Entropy (differences) are an objective quantity which can be measured, there is no subjectivity about it. It is not which parameters you know it is about which parameters you hold fixed.

U, the internal energy is objective. The free energy F = U - TS is the maximum amount of work you can extract from the system. This depends on how much you know about the system. S does indeed depend on what you know about the system. See the Gibbs Paradox for more information.

If two people disagree on the maximum amount of work that could be extracted from a given system (with both of them basing their figure on their own evaluation of S), are there any cases where it would it be impossible to empirically demonstrate that at least the proponent of the lower figure was wrong?

If only changes in S (and F) have measurable consequences, would that not merely mean that assigning an absolute value is an arbitrary choice, which would not mean the same as it being subjective (there could still be an objective conversion between one basis and another, as there is for kinetic energy in different inertial reference frames.)

In the Gibbs Paradox, there is no subjectivity in whether the gases being mixed are the same or different, and no subjectivity in what the change of entropy is in either case. The paradox is that it does not feel right that identity makes an objective difference between the two cases, but the empirically-demonstrable distinction between fermions and bosons shows that this intuition does not hold in general. I believe Von Neumann came up with a QM resolution of the paradox.

Re: It took me 10 years to understand entropy

#208

Post by kgwgk »

Jensson wrote: post #203
kgwgk wrote: post #199

Earlier quoted context omitted.

Have you heard of thermometers? I can have a container with 1l of some gas at room temperature T1 and proceed to heat the room - and the container - to temperature T2. How do you calculate the number of microstates for the sample of gas before and after? How do you think these numbers are related? You said it was easy!

Thermometers is a way to measure temperature, it isn't its theoretical definition. Temperature is defined as energy required per change in entropy. There is no other reasonable way to define temperature, since at its core it measures which way energy flows when two macro systems are connected. Temperature tends to go up as you add energy to things, but not always, for example temperature doesn't go up when you melt i…

> This adds a lot more microstates that wasn't available before, and none of the old microstates are now possible since all old microstates had a total energy level half of what each new microstate has.

As I'm sure you know, the microstates of that sample of gas at some fixed temperature don't have all the same energy. For each temperature there will be a distribution of possible energies. If the temperatures are close enough there will be a large overlap between those distributions.

You cannot just count the microstates of the sample of gas. (You can count microstates of the gas plus reservoir system though.)

Re: It took me 10 years to understand entropy

#209

Post by lubesGordi »

vishnugupta wrote: post #171
oh_my_goodness wrote: post #141

Earlier quoted context omitted.

>This is how I have come to understand entropy. The words disorder and order are a proxy for information content Does information content mean this? ... "How many bits of random-number generator would I need to make the number of micro-states in the macro-state?"

That is my mental model, yes. More bits are needed to capture more detailed (or micro-states as you called it elsewhere in this thread, or finer-grained) information. Let's say there's a stone, we want to know its details. If all we want to know is whether it weighs more than 100KG or not then one bit will do. 1 means > 100KG and 0 means This is just for the storage though; in order to gain the information we need to…

The article does say that some crystalline structures can have more entropy (information) than their fluid state. How could that be? Any ideas on what that fluid state might be? The information content in a crystal is really low.

Re: It took me 10 years to understand entropy

#210

Post by Jensson »

kgwgk wrote: post #208
Jensson wrote: post #203

Earlier quoted context omitted.

Thermometers is a way to measure temperature, it isn't its theoretical definition. Temperature is defined as energy required per change in entropy. There is no other reasonable way to define temperature, since at its core it measures which way energy flows when two macro systems are connected. Temperature tends to go up as you add energy to things, but not always, for example temperature doesn't go up when you melt i…

> This adds a lot more microstates that wasn't available before, and none of the old microstates are now possible since all old microstates had a total energy level half of what each new microstate has. As I'm sure you know, the microstates of that sample of gas at some fixed temperature don't have all the same energy. For each temperature there will be a distribution of possible energies. If the temperatures are clo…

> As I'm sure you know, the microstates of that sample of gas at some fixed temperature don't have all the same energy.

Depends if you do classical statistical physics or quantum. If you do classical they all have the same energy. If you do quantum you have to weight the states according to their probability densities, and the probabilities that the energy deviates are very small which is why classical works fine even when ignoring those.

But quantum statistical physics is way more complex, you should learn the classical statistical physics first before you try to discuss quantum statistical physics. Classical works a lot like the computer science version where you just count states, quantum doesn't.

Post reply on HN