What Is Entropy?

This is the best description of entropy and information I've read: https://arxiv.org/abs/1601.06176

Most of all, it highlights the subjective / relative foundations of these concepts.

Entropy and Information only exist relative to a decision about the set of state an observer cares to distinguish.

It also caused me to change my informal definition of entropy from a negative ("disorder)" to a more positive one ("the number of things I might care to know")

The Second Law now tells me that the number of interesting things I don't know about is always increasing!

This thread inspired me to post it here: https://news.ycombinator.com/item?id=43695358

Surely laws of nature are still relevant since they (presumably) establish limits on how closely a system can be measured and which physical interactions can be simulated by computers (and how accurately).

How can that be axiomatic?

I offer a coherent, concise dissenting view.

Information is the removal of uncertainty. If it does not remove uncertainty it is not information. Uncertainty is state unresolved (potential resolves to state through constructive and destructive interference.)

Entropy is the existential phenomenon of potential distributing over the infinite manifold of negative potential. “Uncertainty.”

Emergence is a potential outcome greater than the capacity found in the sum of any parts.

Modern humanity’s erroneous extrapolations:

- asserting P>=0 without account that in existential reality 0 is the infinite expanse of cosmic void, thus the true mathematical description would be P>=-1

- confuse heat with entropy. Heat is the ultimate universal expression as heat is a product of all work and all existence is winding down (after all). Entropy directs thermodynamics, thermodynamics is not the extent of entropy.

- entropy is NOT the number of possible states in a system. Entropy is the distribution of potential; number of states are boundary conditions which uncalculated potential may reconfigure (the “cosmic ray” or murfy’s rule of component failure.) Existential reality is interference and decay.

- entropy is not “loss”. Loss is the entropy less work achieved.

- this business about “in a closed system “ is an example of how brilliant minds lie to themselves. No such thing exists anywhere accessible by Man. Even theoretically, the principles of decay and the “exogenous” influence of one impercieved influence over a “contained system.” Or “modeled system”, for one self deception is for the scientist or engineer to presume these speak for or on behalf of reality.

Emergence is the potential (the vector space of some capacity) “created” through some system of dynamics (work). “Some” includes the expressive space of all existential or theoretical reality. All emergent potential is “paid for” by burning available potential of some other kind. In nature the natural forces induce work in their extremes. In natural systems these design for the “mitigation of uncertainty” [soft form entropy], aka “intelligence.”

Entropy is the existential phenomenon of potential distributing over negative potential.

Information is the removal of uncertainty. If it does not remove uncertainty, it is not information. (And intelligence is the mitigation of uncertainty.)

Emergence is a potential outcome greater than the capacity found in the sum of any parts.

I guess part of my question is, are there any macrostates that are useful to us that can't be described using more abstract human-subjective terms? If a macrostate can be described using human terms, I'd say the state is somewhat ordered. And if a state can't be described using human terms, then wouldn't it be indistinguishable from "particle soup" and thus not a useful macrostate to talk about?

More precisely, Von Neumann was extending Shannon's information theoretic entropy to quantum channels, which he restated as S(p)=Tr(p ln(p)) - Again showing that information theoretic entropy reveals nothing more about a system than its probability distribution density matrix p.