Demystifying the second law of thermodynamics
erischel.com
Demystifying the second law of thermodynamics
1–10 of 41 posts
Re: Demystifying the second law of thermodynamics
#2It’s probably been 15 years since I looked in detail at a formal treatment of statistical mechanics, so maybe this is well-trodden territory, but... has anyone formalized the idea that the second law can be interpreted as a general inability of computer programs to predict one another? I’m also channeling Wolfram here, but I’m not sure if he ever expressed it exactly this way. But the thing I’m imagining would be: set up some (preferably discrete) dynamical system, and then another different kind of system that you fit variationally to the first. The claim would be that the mutual information between state variables of the second and the first will always gradually decrease — unless they are the exactly the same family, in which case the fit rediscovers the exact dynamics and the two systems are identical. So intuitively, the second law would correspond to the claim that different families of programs cannot accurately simulate each other for very long. In the special case of a coarse graining, this should give you the more familiar story about entropy and the ‘gap’ between macrostates and microstates, but the real story is a more general one than that and is an empirical fact about the “computational universe”, with physics a special case.
Re: Demystifying the second law of thermodynamics
#3This is speaking of entropy and states of gasses.
Isn't there some generalization of entropy, such that even when sentient life-forms tried their damnedest, or even in black-holes, entropy must always trend up? Or was that a mischaracterization?
Re: Demystifying the second law of thermodynamics
#4> you’re probably never going to observe entropy going down in your life
Except when you tidy up your room.
Re: Demystifying the second law of thermodynamics
#5It’s probably been 15 years since I looked in detail at a formal treatment of statistical mechanics, so maybe this is well-trodden territory, but... has anyone formalized the idea that the second law can be interpreted as a general inability of computer programs to predict one another? I’m also channeling Wolfram here, but I’m not sure if he ever expressed it exactly this way. But the thing I’m imagining would be: se…
> has anyone formalized the idea that the second law can be interpreted as a general inability of computer programs to predict one another?
This sounds like the halting problem.
Re: Demystifying the second law of thermodynamics
#6Or you could listen to the Flanders and Swann version: https://www.youtube.com/watch?v=VnbiVw_1FNs
Re: Demystifying the second law of thermodynamics
#7> you’re probably never going to observe entropy going down in your life Except when you tidy up your room.
But you're turning food into poop, and exhaling carbon dioxide, as you do so...
Re: Demystifying the second law of thermodynamics
#8It’s probably been 15 years since I looked in detail at a formal treatment of statistical mechanics, so maybe this is well-trodden territory, but... has anyone formalized the idea that the second law can be interpreted as a general inability of computer programs to predict one another? I’m also channeling Wolfram here, but I’m not sure if he ever expressed it exactly this way. But the thing I’m imagining would be: se…
> has anyone formalized the idea that the second law can be interpreted as a general inability of computer programs to predict one another? This sounds like the halting problem.
Wolfram's "computational irreducibility" is basically the halting problem for physics.
Re: Demystifying the second law of thermodynamics
#9Perhaps the best way I began to understand entropy is when reading about tail inequalities and divergences. In particular, we can view the entropy as a sort of “distance to a uniform distribution.” (Specifically an additive factor away from the KL divergence between the uniform distribution and the distribution under measurement.) Studying the distributions from this perspective yields a bunch of important consequences like, how many samples do you need to tell one distribution from another? Or, how many bits are needed to approximately describe the distribution, from a more general family? Surprisingly, being able to answer these two questions in a precise sense gives you a whole load of tools to fun things like: can an algorithm decide a question with k bits of information? Or can two physical systems diverge in behavior in a meaningful way after T time?
For the (fairly) mathematically inclined reader, I very highly recommend Massart’s Concentration Inequalities and Model Selection.
Re: Demystifying the second law of thermodynamics
#10For some actual demystification for those more computationally inclined. https://www.falstad.com/gas/