Entropy explained, with sheep (2016)
11–20 of 107 posts
Re: Entropy explained, with sheep (2016)
#12For example, there's a well-known phenomenon in probability called concentration of measure. One of the most important examples in computer science is if you flip n coins independently, then the number of heads concentrates very tightly. In particular, the probability that you are more than an epsilon-fraction away from 0.5n heads is at most around e^{-epsilon^2 n}. This is exactly the setting described in the article with the sheep in two pens, and this inequality is used in the design of many important randomized algorithms! A classic example is in load balancing, where random assignment sometimes produces 'nice' configurations that look very much like the high-entropy states described in this article (but unfortunately, many times random assignments don't behave very well, see e.g. the birthday paradox).
The sum of independent random variables is well known to have concentration properties. An interesting question to me is what sorts of other statistics will exhibit these concentration phenomena. An important finding in this area is the bounded differences inequality (https://web.eecs.umich.edu/~cscott/past_courses/eecs598w14/n...), which generally states that any function that doesn't "depend too much" on each individual random argument (and the sum of bounded variables satisfies this assumption) exhibits the same concentration phenomenon. There are some applications in statistical learning, where we can bound the estimation error using certain learning complexity measures that rely on the bounded differences inequality. In the context of this article, that means there's a whole class of statistics that will concentrate similarly, and perhaps exhibit irreversibility at a macroscopic level.
Re: Entropy explained, with sheep (2016)
#13Great explanation. What are the top theories for why the universe began in a low entropy state? The mere fact of that seems to contradict our current understanding of entropy. That implies that there is something very fundamental about the universe which we don't understand.
What part of our understanding does it contradict? The second law of thermodynamics says that entropy increases with time; this seems entirely consistent with a low-entropy past.
One explanation for why the universe began in a low entropy state is that that state has a very low description length. (This is a bit of a truism, since description length is a measure of entropy). But basically, let's just imagine that the universe is a simulation, with an initial state described by an initialization routine that sets up the simulation to run. If the initial state has high entropy, that initialization routine would need to be very long and detailed to describe exactly the location of every electron, neutrino, etc. If the initial state has very low entropy, that initialization routine is very short. If there's a reason to think that a short program is "more probable" than any particular very long program, then that would explain a low-entropy initial condition.
Another explanation is that, if the universe random-walks through all possible configurations, the "past" will still always look lower-entropy than the "future", for any little life-form that occupies that universe, because that life-form's memories will be much more likely to be correlated with the lower-entropy state. (It would have been nice for the article to go into this detail, but it's rarely discussed).
Still another explanation is provided by Many-Worlds interpretation of QM. Again the "big bang" is akin to initializing the wavefunction of the universe to something very simple and compact like a constant function, which as a whole evolves unitarily; the complexity and increasing entropy arises within particular branches of that wavefunction, where an observer requires an ever-longer description length to identify their particular branch.
Re: Entropy explained, with sheep (2016)
#14If the entropy of the universe were to suddenly go in reverse would we be able to detect it or would our memory formation and perceptions being reversed make it indistinguishable from what we experience now?
This is a very good question. Yes, time reversal would also reverse the process of our perception and memory, so we would experience time "moving forward" even if time were "moving" backward. We would experience entropy increasing even if entropy were "decreasing with time". (And it's perfectly valid to call the past "+t" and the future "-t"; the laws of physics don't care; if you do that, you'll see that entropy dec…
[Edit: to expand a bit: time reversal requires that some previous macrostate is achieved again. Entropy decreasing merely requires that the system enters any macrostate represented by less microstates than the current one.
Put in more simple terms, the ice cube could reform but in a different shape. Entropy would have decreased, but it would not "look like" time reversal - it would just look like something very strange had happened. ]
Re: Entropy explained, with sheep (2016)
#15Great article! I'm almost done with a PhD in physics and statistical physocs is still the hardest thing for me to wrap my mind around. Even more so than quantum and relativity. There's something about how unintuitive math becomes in ultra high dimension and the old timey feeling way that it was taught to me (steam engines and stuff) that makes it difficult to learn.
When I took engineering thermodynamics, the focus was on solving "practical" problems, and I never got a good understanding of what entropy was. The point of the class was not to teach physics, though. The point was to get engineers to solve thermodynamics problems (like heat engines and cycles). The primary thing I remember from that course was looking up things in the tables in the back of the book and converting between Btus and other units.
I only understood entropy after taking a statistic physics course. The course's explanation of entropy is basically the same explanation as the article. The point of that class was entirely different than the engineering course, so the professor spent several lectures going into detail about what entropy is and how it relates to everything else. I never had to do any table lookups in that class too.
Side note: you can easily tell what tradition you were taught in based on the sign convention in the first law of thermodynamics you learned:
https://en.wikipedia.org/wiki/First_law_of_thermodynamics#Si...
Re: Entropy explained, with sheep (2016)
#16What am I missing? Is this a weird measure theory thing, where hot objects have even bigger uncountable infinities of states and we get rid them all with something like a change of variables? If you told me spacetime was secretly a cellular automaton I could deal with it, but real numbers ruin everything.
Re: Entropy explained, with sheep (2016)
#17Great explanation. What are the top theories for why the universe began in a low entropy state? The mere fact of that seems to contradict our current understanding of entropy. That implies that there is something very fundamental about the universe which we don't understand.
There is a fantastic book by Leonard Susskind [0] that tackles this questions. His answer - rooted in string theory - is based on a "landscape of possibilities" and we happen to be in the place where all elementary particles (more specifically the Higgs) had the right energy configuration. [0] The cosmic Landscape: https://www.amazon.com/Cosmic-Landscape-String-Illusion-Inte...
Re: Entropy explained, with sheep (2016)
#18Earlier quoted context omitted.
There is a fantastic book by Leonard Susskind [0] that tackles this questions. His answer - rooted in string theory - is based on a "landscape of possibilities" and we happen to be in the place where all elementary particles (more specifically the Higgs) had the right energy configuration. [0] The cosmic Landscape: https://www.amazon.com/Cosmic-Landscape-String-Illusion-Inte...
Where did those particles come from? Is there any scientific theory for the beginning of the beginning? A friend who studied physics told me that there's no reason that matter/energy couldn't have just existed forever. I don't find that explanation satisfying. If "entropy always increases" is a universal rule, then there must have been a time where it went from zero to non-zero. If "matter cannot be created or destro…
In fact, if it is zero at some point in the past, then what happened before that? Did it not increase before that point, or was it less than zero before that point?
Re: Entropy explained, with sheep (2016)
#19The only tiny change I'd like to make is to add a line or two near the end, something along the following lines:
There's a lot fewer ways to arrange water molecules so that they form an ice cube than there are to arrange them as a liquid. Most arrangements of water molecules look like a liquid, and so that's the likely endpoint even if they start arranged as an ice cube.
The same is true of more or less any macroscopic object: the thing that we recognize and name ("chair", "table", "pen", "apple") requires the atoms to remain in one of a fairly small set of particular arrangements. Compared to the vast number of other arrangements of the same atoms ("dust"), the ones where the atoms form the "thing" are quite unlikely. Hence over time it's more likely that we'll find the atoms in one of the other ("random", or "dust-like") arrangements than the one we have a name for. The reason things "fall apart" isn't that there's some sort of preference for it - it's that there are vastly more ways for atoms to be in a "fallen apart" state than arranged as a "thing".Re: Entropy explained, with sheep (2016)
#20Earlier quoted context omitted.
There is a fantastic book by Leonard Susskind [0] that tackles this questions. His answer - rooted in string theory - is based on a "landscape of possibilities" and we happen to be in the place where all elementary particles (more specifically the Higgs) had the right energy configuration. [0] The cosmic Landscape: https://www.amazon.com/Cosmic-Landscape-String-Illusion-Inte...
Where did those particles come from? Is there any scientific theory for the beginning of the beginning? A friend who studied physics told me that there's no reason that matter/energy couldn't have just existed forever. I don't find that explanation satisfying. If "entropy always increases" is a universal rule, then there must have been a time where it went from zero to non-zero. If "matter cannot be created or destro…