Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Entropy explained, with sheep (2016)

aatishb.com

Post reply on HN
Print view View original

91–100 of 107 posts

Re: Entropy explained, with sheep (2016)

#91

Post by aoowii »

Order as humans understand it is different from physical order. We see order as an array of ascending numbers or a house of cards. The universe sees order more like a state of 'useful' energy, where it's possible to extract it, and seemingly became ordered by putting energy in. This is why I think the information theory definition of entropy is a bit misleading.

I like to see it as a radioactive atom, which starts as a useful structure with potential energy that has an inherent timer until this energy is lost due to the universe wanting to return to equilibrium. So it's statistically extremely unlikely to get the atom back to its original energetic state by nature, but not because because it's literally impossible, it's just impossible to do this without putting energy back in and that's not something that happens naturally.

A reversal of entropy is entirely possible, it's just called doing work. Of course, we don't have the knowledge necessary to reverse each individual atom in the melting process of an ice cube, but one day we might. It's theoretically possible with enough work. Of course, there will always be a loss of energy, but i'm pretty sure that's an entirely different thermodynamic law.

Re: Entropy explained, with sheep (2016)

#92

Post by kergonath »

andrewflnr wrote: post #86
kergonath wrote: post #46

Earlier quoted context omitted.

This is a very good question, which goes to the heart of statistical Physics. We use phase spaces for this (typically a 6N-dimensional vector space in which each microstate is represented by a point). The system has a probability of being in (or rather very close to) each microstate, which depends on several factors, like the conditions (isolated system, constant pressure, temperature, number of particles, etc). Coun…

So would it make sense to think of a microstate as a region of phase space, a point and those points "very close to" it? And "increasing number of microstates" just means a larger number of these regions have non-negligible probabilities? In continuous terms you would see this as the distribution flattening out. I might be having trouble visualising what we're integrating, since if it's a probability the integral ove…

Yes, that is the principle. The probability of a single point is zero because an integral over a point is zero, hence “very close to it” (in an infinitesimal volume around the point).

The integral of the probability over the phase space is indeed 1. This is the purpose of the partition function, which is the normalisation factor. The weight function is not normalised a priori.

Re: Entropy explained, with sheep (2016)

#93

Post by anonytrary »

IIAOPSW wrote: post #76
andrewflnr wrote: post #16

So one thing I've never understood is how you can "count" microstates, or bits required to describe them, when the relevant physical parameters all seem to be real numbers. For instance, a gas of N atoms is described by 6N real numbers (3d position and velocity) regardless of how hot it is. The article talks about quanta of energy, but that seems like a simplification at best: a given interaction might be quantized,…

I asked the same question to my professor when I took thermodynamics as an undergrad. In that class we are told that a particle in a box of size 2 can be in twice as many places as a particle in a box of size 1. But in real analysis we learn there are just as many numbers between 0 and 1 as there are between 0 and 2. The answer I was given, in true physicists form, is hand-wave it. There's an intuitive notion that tw…

> energy and position come in discrete units

This is not true; position/time are not quantized in the standard model. String theory is not canonical. I think a better way to think about it is not in terms of size, but in terms of time. A particle in a bigger box, on average, can go on a random walk for longer without hitting a wall. It will take longer for a particle to sufficiently (arbitrarily close) exhaust the phase space of a bigger box.

Re: Entropy explained, with sheep (2016)

#94

Post by raattgift »

jbay808 wrote: post #35
PaulDavisThe1st wrote: post #30

Earlier quoted context omitted.

I think you're priviledging microstates. There are plenty of entropy decreases that are possible without the ice cube reforming. They're (extremely) unlikely of course, but somewhat less unlikely than the new, singular microstate you're asserting is somehow "less of a coincidence". It's vastly more likely (for example) that 1% of the particle velocities reverse than all of them doing so, and that could (depending ...…

Ah, I think we're just miscommunicating. I'm not saying that local time-reversal is more likely to happen than other kinds of entropy decrease; I'm not saying that a system is more likely to retrace its past than to enter a different state of low-entropy. Spontaneous local entropy decreases happen all the time, of course (but are overpowered by entropy increases), and the majority of those won't be exact reversals. A…

> why can't you remember the future the way you remember the past

Memory of a finite brain isn't -- and cannot be -- a perfect record, so to some extent human memory retrodicts personally experienced past events extrapolatively. That human memory is better than human prediction of future events might not be closely coupled to cosmic thermodynamics.

For example, our ancestors with good memories producing more viable offspring than their contemporaries with poor memories but better predictive skills. You might not want to play certain competitive sports against the better predictors but worse rememberers, since they are likely to know where to be to catch the ball or whatnot; but you also might not want to eat the food they prepare because they don't reliably remember crucial food-safety practice.

The relative entropy inside the braincase of a living Australopithecus or H. erectus versus inside a living modern human's braincase has little to do with the change in total entropy of the universe over the past couple million years. It is perfectly plausible under modern physics that humans a couple million years in the future may end up with simpler and smaller brains, rather than larger or more complex ones. And the improving scientific knowledge of the evolutionary changes in human skull dimensions was not helpful in resolving the 20th century question of whether the universe was collapsing, expanding, or static.

Finally we aren't very good at communicating with the other intelligent species on our planet. Maybe orcas or octopuses have terrific memory and don't feel a significant difference between remembering a recent previous hunt and predicting the one they are just about to embark upon. It'd be fun to find out that our distinction between memory and prediction is just another part of human vestigiality, like our inability to produce L-ascorbic acid internally. Colloquially, maybe "future memory" was one of the things that were too metabolically expensive for our starving distant ancestors to live with, and so it fell off like tails.

Re: Entropy explained, with sheep (2016)

#95

Post by raattgift »

PaulDavisThe1st wrote: post #73
jbay808 wrote: post #35

Earlier quoted context omitted.

Ah, I think we're just miscommunicating. I'm not saying that local time-reversal is more likely to happen than other kinds of entropy decrease; I'm not saying that a system is more likely to retrace its past than to enter a different state of low-entropy. Spontaneous local entropy decreases happen all the time, of course (but are overpowered by entropy increases), and the majority of those won't be exact reversals. A…

>why can't you remember the future the way you can remember the past this is getting a bit meta, maybe even off-topic. But I think this is fairly simple to explain: you can't remember states you haven't been in. Generalized memory implies some record of a state that has occured - states that have not occured cannot be remembered. The problem is that memory of types that we are familiar with (human, written, computer)…

> You can't remember states you haven't been in

If only memory worked that way!

False memories are commonplace. Where exactly did you put your keys? Even if you give the right location, is that a true memory of the state you and the keys were in when you separated, or is it a retrodiction ("I probably put them in their usual storage place")?

> A memory system that operates on the timescales of ...

You mean one that measures parts of the world periodically?

ISTR we discussed this some exp(10^120) years ago[1] but I forget whether we reached any conclusions.

- --

[1] Dyson, Kleban & Susskind, 2002, https://arxiv.org/abs/hep-th/0208013 eqn (5.2).

Re: Entropy explained, with sheep (2016)

#96

Post by PaulDavisThe1st »

raattgift wrote: post #95
PaulDavisThe1st wrote: post #73

Earlier quoted context omitted.

>why can't you remember the future the way you can remember the past this is getting a bit meta, maybe even off-topic. But I think this is fairly simple to explain: you can't remember states you haven't been in. Generalized memory implies some record of a state that has occured - states that have not occured cannot be remembered. The problem is that memory of types that we are familiar with (human, written, computer)…

> You can't remember states you haven't been in If only memory worked that way! False memories are commonplace. Where exactly did you put your keys? Even if you give the right location, is that a true memory of the state you and the keys were in when you separated, or is it a retrodiction ("I probably put them in their usual storage place")? > A memory system that operates on the timescales of ... You mean one that m…

I was commenting on a comment that specifically disconnected the word "memory" from the specifics of human mental memory.

Re: Entropy explained, with sheep (2016)

#98

Post by andrewflnr »

kergonath wrote: post #92
andrewflnr wrote: post #86

Earlier quoted context omitted.

So would it make sense to think of a microstate as a region of phase space, a point and those points "very close to" it? And "increasing number of microstates" just means a larger number of these regions have non-negligible probabilities? In continuous terms you would see this as the distribution flattening out. I might be having trouble visualising what we're integrating, since if it's a probability the integral ove…

Yes, that is the principle. The probability of a single point is zero because an integral over a point is zero, hence “very close to it” (in an infinitesimal volume around the point). The integral of the probability over the phase space is indeed 1. This is the purpose of the partition function, which is the normalisation factor. The weight function is not normalised a priori.

That helps a lot. Thanks!

Re: Entropy explained, with sheep (2016)

#99

Post by jbay808 »

PaulDavisThe1st wrote: post #73
jbay808 wrote: post #35

Earlier quoted context omitted.

Ah, I think we're just miscommunicating. I'm not saying that local time-reversal is more likely to happen than other kinds of entropy decrease; I'm not saying that a system is more likely to retrace its past than to enter a different state of low-entropy. Spontaneous local entropy decreases happen all the time, of course (but are overpowered by entropy increases), and the majority of those won't be exact reversals. A…

>why can't you remember the future the way you can remember the past this is getting a bit meta, maybe even off-topic. But I think this is fairly simple to explain: you can't remember states you haven't been in. Generalized memory implies some record of a state that has occured - states that have not occured cannot be remembered. The problem is that memory of types that we are familiar with (human, written, computer)…

Sorry for the late reply. Don't know if you'll still be reading this, but...

> This is getting a bit meta, maybe even off-topic. But I think this is fairly simple to explain: you can't remember states you haven't been in. Generalized memory implies some record of a state that has occured - states that have not occured cannot be remembered.

In a discussion about the arrow of time, this is somewhat begging the question! The difference between a state that you've "been in" and a state that you "will be in" is exactly the subject that we're discussing. How do you precisely define "been in" or "occurred" in a way that doesn't reverse under a transformation from t -t?

The answer I've provided elsewhere in this thread is along the lines of: Other than melting icecubes and scrambling eggs, the only other difference you can notice between the past and the future is that you can remember the past, but you cannot remember the future. If you could remember the future just as well as you remember the past, you probably wouldn't have strong opinions about which way time goes (or which direction is "clockwise"). If you, Merlin-like, could only remember the future then you'd probably be here asking why you always observe entropy decreasing in closed systems, and complaining that they got the second law of thermodynamics backwards.

On a fine-grained scale causality works just as well in a rewound video, although it is full of spontaneous-seeming coincidences with surprising macroscopic effects. There are only two things that establish an arrow of time: * The universe has an entropy gradient along the "time axis", with one direction (which we can call 'P') having lower entropy and the other ('F') having higher entropy. * We (and other physical systems) are able to form memories in one direction, but not the other. Because of this, we perceive a sense that time progresses "from" the direction that we can remember. This happens to be the direction of increasing entropy. Because of this ability to remember only along one direction of the entropy gradient, we call 'P' the past and 'F' the future.

This is not a coincidence. Memory operates on systems of increasing entropy, so you'll always only remember the past having less entropy than the present. [1]

[1]: https://phys.org/news/2009-08-physicist-solution-arrow-of-ti...

Re: Entropy explained, with sheep (2016)

#100

Post by kgwgk »

roimor wrote: post #97

I don't really understand this "Entropy Is All About Arrangements" takeaway. A fair coin has higher entropy than a biased one. What are the "arrangements" in this case?

The coins don't have entropy, the sequences they produce do (in information theory sense, this is not about physics).

For a long sequence (say 1000), the former will produce close to 500 heads and 500 tails. The latter, assuming one heads are three times as probable as tails, will produce around 750 heads and 250 tails. There are many more different sequences of the first kind.

Post reply on HN