Live data from Hacker News

Entropy explained, with sheep (2016)

aatishb.com

51–60 of 107 posts

Re: Entropy explained, with sheep (2016)

#51
post #39
post #28

Earlier quoted context omitted.

A higher entropy state has a longer description length. For example, let's say I have a magic electron microscope that can scan and record the exact position and velocity of each particle in some 1-cubic-micron volume, to within Heisenberg uncertainty limits and some finite digitization precision. If my sample is a 1-cubic-micron volume of flawless monocrystalline silicon at 0 Kelvin, I can 'zip' my recording and tra…

Your example of monocrystalline silicon at (almost) 0 Kelvin has actually higher entropy than your example of saltwater.

Can you elaborate? And what if I used as comparison something like room-temperature doped polysilicon?

Re: Entropy explained, with sheep (2016)

#52
post #42

That still doesn’t answer the question how, if the laws of physics are time-symmetric, the universe as a whole can have a time-asymmetric evolution of entropy. I.e., if something forces entropy to increase in the long run, then that should hold in both directions of time. So what is it that causes entropy to only increase in the direction of the future, but not in the direction of the past, given that the laws pf phy…

We don’t have an answer to this question. I don’t want to discuss metaphysics here, but there is a very interesting discussion on that subject here: https://youtube.com/watch?v=-6rWqJhDv7M

Re: Entropy explained, with sheep (2016)

#53

This was really great. It's how I was taught entropy at college (biophysics and molecular biology) though without the sheep. "Statistical Mechanics" was the name our professor used. The only tiny change I'd like to make is to add a line or two near the end, something along the following lines: There's a lot fewer ways to arrange water molecules so that they form an ice cube than there are to arrange them as a liquid.…

This kind of intuition can be dangerous:

"Students who believe that spontaneous processes always yield greater disorder could be somewhat surprised when shown a demonstration of supercooled liquid water at many degrees below 00 C. The students have been taught that liquid water is disorderly compared to solid ice. When a seed of ice or a speck of dust is added, crystallization of some of the liquid is immediate. Orderly solid ice has spontaneously formed from the disorderly liquid.

"Of course, thermal energy is evolved in the process of this thermodynamically metastable state changing to one that is stable. Energy is dispersed from the crystals, as they form, to the solution and thus the final temperature of the crystals of ice and liquid water are higher than originally. This, the instructor ordinarily would point out as a system-surroundings energy transfer. However, the dramatic visible result of this spontaneous process is in conflict with what the student has learned about the trend toward disorder as a test of spontaneity.

"Such a picture might not take a thousand words of interpretation from an instructor to be correctly understood by a student, but they would not be needed at all if the misleading relation of disorder with entropy had not been mentioned."

http://entropysite.oxy.edu/cracked_crutch.html

Re: Entropy explained, with sheep (2016)

#54
post #42

That still doesn’t answer the question how, if the laws of physics are time-symmetric, the universe as a whole can have a time-asymmetric evolution of entropy. I.e., if something forces entropy to increase in the long run, then that should hold in both directions of time. So what is it that causes entropy to only increase in the direction of the future, but not in the direction of the past, given that the laws pf phy…

We don’t have an answer to this question. I don’t want to discuss metaphysics here, but there is a very interesting discussion on that subject here: https://youtube.com/watch?v=-6rWqJhDv7M

Let me rephrase maybe: Given the state of affairs I described above, I don’t understand what is the convincing argument that entropy does indeed increase in the long run. Any argument given should also work in the reverse direction, given the symmetry of time, shouldn’t it? (And thereby create a kind of reductio ad absurdum.) If not, why not?

Re: Entropy explained, with sheep (2016)

#55
post #24

Earlier quoted context omitted.

Your reasoning is strange. Actually, higher entropy is what we may call "of lower complexity" requiring ever-shorter description length.

If you care about describing the details, you can compress your description better if it's a low-entropy state. But of course, cosmology is full of more mundane explanations about how the limit of the possible entropy of the universe can grow with time, so a high-entropy state suddenly has a lot of room to increase even further.

That's a good point. I was going to mention expansion of the universe as another one, but that invites its own line of "why" questions!

Re: Entropy explained, with sheep (2016)

#56
post #53

This was really great. It's how I was taught entropy at college (biophysics and molecular biology) though without the sheep. "Statistical Mechanics" was the name our professor used. The only tiny change I'd like to make is to add a line or two near the end, something along the following lines: There's a lot fewer ways to arrange water molecules so that they form an ice cube than there are to arrange them as a liquid.…

This kind of intuition can be dangerous: "Students who believe that spontaneous processes always yield greater disorder could be somewhat surprised when shown a demonstration of supercooled liquid water at many degrees below 00 C. The students have been taught that liquid water is disorderly compared to solid ice. When a seed of ice or a speck of dust is added, crystallization of some of the liquid is immediate. Orde…

I'd rather they get more curious about the spontaneous crystallization, and get inspired to look deeply into what's really going on when that happens.

Re: Entropy explained, with sheep (2016)

#58
post #44
post #12

Loved this article. There are so many applications of entropy and statistical physics in computer science, and I find it fascinating that the same general properties are useful in such different contexts. For example, there's a well-known phenomenon in probability called concentration of measure. One of the most important examples in computer science is if you flip n coins independently, then the number of heads conc…

There's a related anecdote about John von Neumann: he used to joke that he has superpowers and can easily tell truly random and pseudo random sequences apart. He asked people to sit down in another room and generate a 0/1 sequence via coin flips, and record it. Then, generate another sequence by heart, trying to mimick randomness as much as possible. When people finally showed the two sequences to him, Neumann could…

That's a cool anecdote :-) I wouldn't say it uses concentration of measure exactly, but I see how it is related. The anecdote is about asymptotic properties of random sequences, and concentration of measure is about the same too. In this case, I think you can show that homogenous blocks of length log(n) - log log (n) occur at least with constant probability as n gets large. In other words, the length of homogenous blocks is basically guaranteed to grow with n. I suppose a human trying to generate a random sequence will prevent homogenous blocks above a certain constant length from appearing regardless of the length of the sequence, which would make distinguishing the sequences for large n quite easy!

I think there is also a quite strong connection in this anecdote to the information-theoretic notion of entropy, which takes us all the way back to the idea of entropy as in the article :-) Information-theoretically, the entropy of a long random sequence concentrates as well (it concentrates around the entropy of the underlying random variable). The implication is that with high probability, a sampled long random sequence will have an entropy close to a specific value.

Human intuition actually is somewhat correct in the anecdote, though! The longer the homogenous substring, the less entropy the sequence has, and the less likely it is to appear (as a limiting example, the sequence of all 0s or all 1s is extremely ordered, but extremely unlikely to appear). I think where it breaks down is that there are sequences with relatively long homogenous substrings with entropy close to the specific values (in the sense that the length is e.g. log (n) - log log (n) as in the calculation before), where the human intuition of the entropy of the sequence is based on local factors (have I generated 'too many' 0s in a row?) and leads us astray.

Re: Entropy explained, with sheep (2016)

#59
post #47

I love how the notion of entropy permeates into so many other things. It's fundamental, universal, and at the heart of nearly every aspect of our existence. Take philosophy. If the ultimate state of everything culminates in chaos (according to the theory of entropy), the human existence constitues the exact opposite: controlling the chaos that surrounds us, and shaping it into something useful and, in entropy-speak,…

Alternatively, all that attempts to control chaos and decrease entropy actually results in faster entropy increase overall on a systemic level. I remember reading about a (Russian?) physicist that believed that life simply happens as a result of the universe's attempt to increase entropy faster on sufficiently complicated systems. If someone remembers his name I'd be obliged.

Dorion Sagan (Yes, Carl Sagan's son) also covers this in his book Into The Cool. That life is only a force multiplier in increasing it universally, basically.

Re: Entropy explained, with sheep (2016)

#60
post #42

That still doesn’t answer the question how, if the laws of physics are time-symmetric, the universe as a whole can have a time-asymmetric evolution of entropy. I.e., if something forces entropy to increase in the long run, then that should hold in both directions of time. So what is it that causes entropy to only increase in the direction of the future, but not in the direction of the past, given that the laws pf phy…

The answer I've provided elsewhere in this thread is along the lines of: Other than melting icecubes and scrambling eggs, the only other difference you notice between the past and the future is that you can remember the past, but you cannot remember the future. If you could remember the future just as well as you remember the past, you probably wouldn't have strong opinions about which way time goes (or which direction is "clockwise"). If you, Merlin-like, could only remember the future then you'd probably be here asking why you always observe entropy decreasing in closed systems.

But memory operates on systems of increasing entropy, so you'll always only remember the past having less entropy. [1]

(Edit: citation)

[1] https://phys.org/news/2009-08-physicist-solution-arrow-of-ti...

Post reply on HN