Live data from Hacker News

It took me 10 years to understand entropy

cantorsparadise.com

121–130 of 295 posts

Re: It took me 10 years to understand entropy

#121
post #107

Earlier quoted context omitted.

I think he is confusing the usage of entropy in physics and computer science. In computer science entropy is conditional probability and depends on what we know about a system.

As it does in physics! "which parameters [thermodynamic variables] you know" ~ "which parameters [thermodynamic variables] you hold fixed" (or know in average, like the energy for a system in a heat bath where the temperature is fixed) https://bayes.wustl.edu/etj/articles/theory.1.pdf http://nicf.net/articles/thermodynamics-statistical-mechanic...

You can observe the movement of molecules beyond macro properties like temperature.

Re: It took me 10 years to understand entropy

#122
The typical measure of entropy (Shannon or Gibbs, and let's spare details for later and after you've read up on the theory of large deviations) is

- sum (p log(p))

which is not that different than the formula for the mean

sum (p 1/n)

the critical difference is the normalization constant is based on the probability of the state rather than assuming a uniform probability over all states.

So, in effect, the entropy is a measure of the mean. It is a measure adopted to the case where "mean" is ill-defined because the number of modes and/or the variation around those modes is not handled well by simpler metrics.

Re: It took me 10 years to understand entropy

#123
post #107

Earlier quoted context omitted.

As it does in physics! "which parameters [thermodynamic variables] you know" ~ "which parameters [thermodynamic variables] you hold fixed" (or know in average, like the energy for a system in a heat bath where the temperature is fixed) https://bayes.wustl.edu/etj/articles/theory.1.pdf http://nicf.net/articles/thermodynamics-statistical-mechanic...

You can observe the movement of molecules beyond macro properties like temperature.

Sure, at least in principle. And if you knew what every molecule was doing the entropy would vanish.

Re: It took me 10 years to understand entropy

#124
I had the pain, and pleasure, of taking and then (assistant) teaching thermodynamics at MIT.

One of the tidbits that always stuck with me was that astronomers have estimated that observable universe’s total entropy:

When you compare that value to the maximum possible entropy, i.e. the heat death of the universe, and then to the ridiculously low entropy state of the beginning of the universe, we are currently halfway along in that ‘timeline’.

It always brought to mind a grandfather clock; the clock stops when the weight hits the floor, and we are halfway there…

Re: It took me 10 years to understand entropy

#125
The Science of Can and Can't[1] is interesting in how it looks to address a number of fundamentals via counterfactuals including the 2nd Law of Thermodynamics.

Edit: See [2] for background about Constructor Theory.

[1] https://www.chiaramarletto.com/books/the-science-of-can-and-... [2] https://www.youtube.com/watch?v=8DH2xwIYuT0

Re: It took me 10 years to understand entropy

#126

I don't understand entropy and this article did not change it. The issue I take is with the definition of "the most likely state". Think of a series of random bits that can be either 0 or 1 with equal probability. How likely is it that they are all 0 or all 1? Not very likely. There is exactly one configuration. How likely is it that they have a specific configuration of 0 and 1? Equally likely. All states are equall…

First, entropy is a macroscopic property, it makes no sense to talk about the entropy of a single particle. Second, entropy is not a fundamental property, it depends on what the observer cares about. Take the common example of a gas in the corner of a box, in that case we care about the density distribution in the box, a macroscopic property. To make this more concrete, one way to quantify the density distribution could be to tessellate the box with cubes of some size, find the ones with the lowest and highest density, and use the difference between those two densities as a measure of density variation.

Once we have decided that all we care about is the density variation in the box, we can go through all possible microscopic states and group them together by their density variation. Finally a low entropy macroscopic state is simply a macroscopic state - a certain density variation - for which there are only a few microscopic states that have the corresponding density variation. On the other hand a high entropy macroscopic state is a macroscopic state for which there are many microscopic states that have the corresponding density variation. You can also call the microscopic states low or high entropy but only with reference to the macroscopic property you use to group them, in themselves microscopic states are not low or high entropy.

If you observe a low entropy macroscopic state, then you know a lot about the microscopic state, after all there are only very few. If you observe a high entropy macroscopic state, then you know a lot less about the microscopic state, there are much more possibilities even though they are microscopically indistinguishable. And if there are no limiting constraints on how the microscopic states can evolve, if the evolution is essentially a random walk through all possible microscopic states, then the entropy of the system will increase with high probability as it is much more probable to randomly walk into one of the many microscopic states associated with a high entropy macroscopic state than to walk into one of the few microscopic states associated with a low entropy macroscopic state.

Re: It took me 10 years to understand entropy

#127
> Boltzmann imagined that our universe could have reached thermodynamical equilibrium and its maximal entropy state a long time ago, but that a spontaneous entropy decrease to the level of our early universe occured after an extremely long period of time, just for statistical reasons.

I’m interested in reading more about this. Any pointers?

Re: It took me 10 years to understand entropy

#129
scientific method: it started with thermodynamic entropy, but scientits found out that this truth is much deeper engrained in our universe, then we got a mathematically generalized version, which is now used used to explain the "arrow of time" which our time reversable physics equations would not be able to explain alone.

Re: It took me 10 years to understand entropy

#130

Earlier quoted context omitted.

"Ultraviolet Catastrophe"'s problem is that it sounds way cooler than it is. I mean, that's the title of a cyberpunk novel; more hyperbolic (and disappointing) than pretentious. Eigenthings would be second on my list of pretentiousness (oddly, not gedankenthings; I'm inconsistent I guess). Standard Model names strike me whimsical to the point of being undignified.

Ultraviolet catastrophe is cool! It’s why we needed quantum physics, otherwise there would be catastrophic infinite energy emerging from random high frequency oscillations!

Well that's not quite what it is, first of all, and second of all I don't like the practice of calling something a catastrophe when its just predicted by a theory, but didn't happen. I mean, bad theories always predict something horrible should be happening. What's next, do flat earthers get to have an "ocean water catastrophe" because their theory implies all the water drains away? Or a "super luminal" catastrophe because their theory requires infinite unending linear acceleration of the earth (and moon and sun)? I mean, it really is a catastrophe I guess...for the theorist.
Post reply on HN