Live data from Hacker News

It took me 10 years to understand entropy

cantorsparadise.com

51–60 of 295 posts

Re: It took me 10 years to understand entropy

#51
post #39

Earlier quoted context omitted.

OP is about as good as any explanation I've seen.

"Compressibility" (in the software sense) would perhaps carry the most meaning. Entropy is ultimately all about the ability to extract information (i.e. work) out of a system.

Indeed, it was actually James Gleick's book The Information that helped me understand the concept better.

Re: It took me 10 years to understand entropy

#52

The author mentions Boltzman brains and that a human body could theoretically spontaneously form out of particles given a long enough time span. Of course, nothing like this can ever happen. It’s the fallacy of thinking infinite time means infinite possibilities.

It's not a fallacy, it's a paradox. One that indicates that our theories of quantum fluctuations in an infinite universe are incomplete.

This podcast episode has a good discussion on the subject: https://universetoday.fireside.fm/745

Re: It took me 10 years to understand entropy

#53
post #33
post #25

One aspect of entropy that I always find counterintuitive is that unlike mass, charge, etc. it is not a physical quantity. In fact, from the point of view of an experimenter with perfect information about a physical system, the entropy of the system is exactly conserved over time (as made precise by Liouville's Theorem). The Second Law survives in this setting only in the most trivial sense that a constant function d…

Not really. Even from a point of view of an experimenter with perfect information, the entropy of the system declines over time as fewer and fewer bits are needed to describe the system. For example, start with a Glas of warm water and an ice cube in it. Over time, the ice will melt and the range of different temperatures of the molecules decline. Consequently, you need fewer and fewer bits to describe the complete s…

Because ice is solid you can argue it takes less information because the particles aren't moving at all or in together in unison, so it will take less buts to encode.

Furthermore, velocity is also a product of direction as much as speed, so if you take into consideration a solid object may vibrate it's particles in the same direction while a liquid can have it's particles in infinite directions, you're talking about way more information you have to encode.

Re: It took me 10 years to understand entropy

#54
post #25

One aspect of entropy that I always find counterintuitive is that unlike mass, charge, etc. it is not a physical quantity. In fact, from the point of view of an experimenter with perfect information about a physical system, the entropy of the system is exactly conserved over time (as made precise by Liouville's Theorem). The Second Law survives in this setting only in the most trivial sense that a constant function d…

The way to make sense of entropy is to treat it as a subjective quantity. A subjective quantity is a function where the observer's state of knowledge is one of the input arguments.

The article describes it as a measure of hidden information in a system, which is a good description. But that's not a property of the system itself, it's a property of the observer, from whom the information is hidden.

So different observers with different information about a system will have different opinions about its entropy.

My password, for example, to me has zero entropy. I know its microstate. But it's quite secure from someone trying to guess it, and they will think it's quite high in entropy.

If all you know about a system is that it's a kilogram of air at room temperature, it will seem quite high in entropy to you, as many possible microstates are consistent with that description. But if you have godlike knowledge of the exact configuration of every particle in the container, it will seem very low in entropy to you, and that's more than just an accounting difference. Indeed you can use that information to operate a Maxwell's demon and turn the system into a heat engine, splitting the cold and hot molecules into separate spaces and extracting work as though the system really had low entropy to start with. Because it did. To you.

Most of the confusion about entropy comes from what Jaynes calls the mind projection fallacy: the tendency to treat our uncertainty about a system as a property of the system, rather than a property of ourselves.

Re: It took me 10 years to understand entropy

#55

The author mentions Boltzman brains and that a human body could theoretically spontaneously form out of particles given a long enough time span. Of course, nothing like this can ever happen. It’s the fallacy of thinking infinite time means infinite possibilities.

Why is it not bound to happen eventually?

Re: It took me 10 years to understand entropy

#56
post #25

One aspect of entropy that I always find counterintuitive is that unlike mass, charge, etc. it is not a physical quantity. In fact, from the point of view of an experimenter with perfect information about a physical system, the entropy of the system is exactly conserved over time (as made precise by Liouville's Theorem). The Second Law survives in this setting only in the most trivial sense that a constant function d…

Entropy in thermodynamics is a statistical effect which acts like a "force" because of the immense number of particles and sub-states in play. A perfect simulation of gas particles bouncing in a two-chamber system will result in the "pressure" equalising because that is overwhelmingly the most likely state to end up in.

To be honest, I hadn't heard of Louisville's Theorem before but it doesn't seem to imply what you're saying -- in fact it is used to prove the fluctuation theorem which quantifies the probability of entropy spontaneously decreasing (as thermodynamic entropy is a statistical effect).

Re: It took me 10 years to understand entropy

#57
post #49
post #46

Earlier quoted context omitted.

> that a human body could theoretically spontaneously form out of particles given a long enough time span. To be fair, isn't this precisely what happened?

If you leave out "spontaneously"!

Specifically if you leave it in! It's a matter of viewpoint or scope.

"Spontaneous" can be defined a few different ways: https://www.merriam-webster.com/dictionary/spontaneous

From the link "2: arising from a momentary impulse" you get the implied meaning from the original comment if you assume standard human experience of a "moment" IE a few seconds or less.

However you could argue that human evolution is but a moment in the scope of the universe.

And then with definition "5: developing or occurring without apparent external influence, force, cause, or treatment"

The only way it wouldn't be spontaneous would be if an external actor (Deity of your choice?) directed human evolution somehow. To say this is debatable is an understatement..

And so we have fun in word play and hopefully appreciation of different viewpoints.

:-)

edit: rearranged for better flow

Re: It took me 10 years to understand entropy

#58
post #25

One aspect of entropy that I always find counterintuitive is that unlike mass, charge, etc. it is not a physical quantity. In fact, from the point of view of an experimenter with perfect information about a physical system, the entropy of the system is exactly conserved over time (as made precise by Liouville's Theorem). The Second Law survives in this setting only in the most trivial sense that a constant function d…

If I can pick your brain, there’s a related concept — entropy production. How does that relate with these ideas?

Not the GP, but entropy production is any process that increases your ignorance about a system (and usually, if you're doing it intentionally, everybody else's ignorance as well).

To produce entropy you have to grow the number of possible microstates that are consistent with available knowledge of the macrostate.

Usually you accomplish that by converting stored energy into heat somehow. A charged capacitor has low entropy compared for the energy it holds; discharge it through a resistor, and you produce a bunch of entropy because that energy can now be distributed in a lot more ways among a lot more degrees of freedom, and nobody can possibly keep track of them.

Re: It took me 10 years to understand entropy

#59
I thought entropy (in the Shannon sense) was a property of discrete and finite probability distributions. It's essentially a measure of how random a sample from such a probability distribution is. Notably, continuous probability distributions don't have meaningful entropy (or in some sense, their entropy is always infinite). It's worth considering the similarities and differences between entropy and standard deviation.

I thought the 2nd law of thermodynamics was saying that with incomplete knowledge, the probability distribution of possible states becomes more and more spread out as time goes on. It's almost a limit to how you can make predictions or simulations of physics when the initial state of the system is not fully known. Equivalently, it's a banal statement about chaos in the sense of chaos theory.

The only thing I don't get is how physicists get around the discrete and finite restriction. Maybe the state of the system is not what has entropy. Rather, one can define an arbitrary function f from the system to a finite set S, and then talk about the entropy of f(System at time t), because this is indeed a discrete and finite probability distribution which you can take the entropy of.

Hmmm. Maybe I understand entropy.

Re: It took me 10 years to understand entropy

#60
post #46

The author mentions Boltzman brains and that a human body could theoretically spontaneously form out of particles given a long enough time span. Of course, nothing like this can ever happen. It’s the fallacy of thinking infinite time means infinite possibilities.

> that a human body could theoretically spontaneously form out of particles given a long enough time span. To be fair, isn't this precisely what happened?

"Oh, that was easy," says Man, and for an encore goes on to prove that black is white and gets himself killed on the next zebra crossing."
Post reply on HN