What Is Entropy?
johncarlosbaez.wordpress.com
What Is Entropy?
1–10 of 221 posts
Re: What Is Entropy?
#2Entropy is the distribution of potential over negative potential.
This could be said "the distribution of what ever may be over the surface area of where it may be."
This is erroneously taught in conventional information theory as "the number of configurations in a system" or the available information that has yet to be retrieved. Entropy includes the unforseen, and out of scope.
Entropy is merely the predisposition to flow from high to low pressure (potential). That is it. Information is a form of potential.
Philosophically what are entropy's guarantees?
- That there will always be a super-scope, which may interfere in ways unanticipated;
- everything decays the only mystery is when and how.
Re: What Is Entropy?
#3A well known anecdote reported by Shannon:
"My greatest concern was what to call it. I thought of calling it 'information,' but the word was overly used, so I decided to call it 'uncertainty.' When I discussed it with John von Neumann, he had a better idea. Von Neumann told me, 'You should call it entropy, for two reasons. In the first place your uncertainty function has been used in statistical mechanics under that name, so it already has a name. In the second place, and more important, no one really knows what entropy really is, so in a debate you will always have the advantage.'"
See the answers to this MathOverflow SE question (https://mathoverflow.net/questions/403036/john-von-neumanns-...) for references on the discussion whether Shannon's entropy is the same as the one from thermodynamics.
Re: What Is Entropy?
#4Entropy is the distribution of potential over negative potential. This could be said "the distribution of what ever may be over the surface area of where it may be." This is erroneously taught in conventional information theory as "the number of configurations in a system" or the available information that has yet to be retrieved. Entropy includes the unforseen, and out of scope. Entropy is merely the predisposition…
All definitions of entropy stem from one central, universal definition: Entropy is the amount of energy unable to be used for useful work. Or better put grammatically: entropy describes the effect that not all energy consumed can be used for work.
Re: What Is Entropy?
#5Entropy is the distribution of potential over negative potential. This could be said "the distribution of what ever may be over the surface area of where it may be." This is erroneously taught in conventional information theory as "the number of configurations in a system" or the available information that has yet to be retrieved. Entropy includes the unforseen, and out of scope. Entropy is merely the predisposition…
> Entropy includes the unforseen, and out of scope.
Mmh, no it doesn't. You need to define your state space, otherwise it's an undefined quantity.
Re: What Is Entropy?
#6Entropy is the distribution of potential over negative potential. This could be said "the distribution of what ever may be over the surface area of where it may be." This is erroneously taught in conventional information theory as "the number of configurations in a system" or the available information that has yet to be retrieved. Entropy includes the unforseen, and out of scope. Entropy is merely the predisposition…
All definitions of entropy stem from one central, universal definition: Entropy is the amount of energy unable to be used for useful work. Or better put grammatically: entropy describes the effect that not all energy consumed can be used for work.
There's a good case to be made that the information-theoretic definition of entropy is the most fundamental one, and the version that shows up in physics is just that concept as applied to physics.
Re: What Is Entropy?
#7Entropy is the distribution of potential over negative potential. This could be said "the distribution of what ever may be over the surface area of where it may be." This is erroneously taught in conventional information theory as "the number of configurations in a system" or the available information that has yet to be retrieved. Entropy includes the unforseen, and out of scope. Entropy is merely the predisposition…
Baez seems to use the definition you call erroneous: "It’s easy to wax poetic about entropy, but what is it? I claim it’s the amount of
information we don’t know about a situation, which in principle we could learn."
Re: What Is Entropy?
#8Earlier quoted context omitted.
All definitions of entropy stem from one central, universal definition: Entropy is the amount of energy unable to be used for useful work. Or better put grammatically: entropy describes the effect that not all energy consumed can be used for work.
There's a good case to be made that the information-theoretic definition of entropy is the most fundamental one, and the version that shows up in physics is just that concept as applied to physics.
My favorite course I took as part of my physics degree was statistical mechanics. It leaned way closer to information theory than I would have expected going in, but in retrospect should have been obvious.
Unrelated: my favorite bit from any physics book is probably still the introduction of the first chapter of "States of Matter" by David Goodstein: "Ludwig Boltzmann, who spent much of his life studying statistical mechanics, died in 1906, by his own hand. Paul Ehrenfest, carrying on the work, died similarly in 1933. Now it is our turn to study statistical mechanics."
Re: What Is Entropy?
#9I really liked the approach my stat mech teacher used. In nearly all situations, entropy just ends up being the log of the number of ways a system can be arranged (https://en.wikipedia.org/wiki/Boltzmann%27s_entropy_formula) although I found it easiest to think in terms of pairs of dice rolls.
Re: What Is Entropy?
#10Entropy is the distribution of potential over negative potential. This could be said "the distribution of what ever may be over the surface area of where it may be." This is erroneously taught in conventional information theory as "the number of configurations in a system" or the available information that has yet to be retrieved. Entropy includes the unforseen, and out of scope. Entropy is merely the predisposition…
> Entropy includes the unforseen, and out of scope. Mmh, no it doesn't. You need to define your state space, otherwise it's an undefined quantity.
But it is possible to account for the unforseen (or out-of-vocabulary) by, for example, a Good-Turing estimate. This satisfies your demand for a fully defined state space while also being consistent with GP's definition.