Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Entropy Explained, with Sheep (2016)

engineersedge.com

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Entropy Explained, with Sheep (2016)

#11

Post by throwuxiytayq »

kaonwarb wrote: post #10

I appreciate the explanation, but the very first example doesn't sit well with me. Water forming into ice cubes spontaneously looks weird simply because we’re not used to seeing it. Consider a time-lapse of an icicle forming as a sort of counter-example: https://m.youtube.com/watch?v=mmHQft7-iSU (Not refuting entropy as the order of time at all, just noting a visual example is not great evidence.)

Ah, but the icicle isn’t really equivalent to water undripping and refreezing back into a nice cube-shaped object, at presumed room temperature (since our point of reference is the first gif). That would be weird to see IRL always, no matter what. You could watch that gif a million times and you’d still shit your pants if that happened to the glass of water on your desk.

Re: Entropy Explained, with Sheep (2016)

#12

Post by IncreasePosts »

xtrapol8 wrote: post #7
IncreasePosts wrote: post #6

Earlier quoted context omitted.

Consider that if you Google for `"resistive variance" entropy`, the only hit is for the hn comment. It doesn't make sense because what they wrote makes no sense. Probably some looney with their own definition of entropy.

Well, this looney likes to point out that the manifold surface area (which is not always uniform) determines the rate of or density of distribution. All this is accounted for in the math selected for whichever for instance (“number of possible states” would count them) though a superior definition is one which is the most general technically correct without enumerating exceptions or extraneous clauses. Resistance con…

I'm not interested in reading your word salad.

Re: Entropy Explained, with Sheep (2016)

#13

Post by kelnos »

IncreasePosts wrote: post #6
ninetyninenine wrote: post #5

Earlier quoted context omitted.

>Entropy is a fancy word for potential distribution over negative potential. Negative potential is the “surface area” over which potential may distribute. I don't understand this. Please elucidate.

Consider that if you Google for `"resistive variance" entropy`, the only hit is for the hn comment. It doesn't make sense because what they wrote makes no sense. Probably some looney with their own definition of entropy.

Your post is a good primer on bullshit detection: if you read something on the internet that sounds confidently authoritative, but you yourself are not well-versed in the subject, find what seems to be some key phrases and search the web for them. If you find lots of other hits on seemingly-reputable sources, then what you've read may be correct. If you only find the thing that you've just read, it's bullshit, with high probability.

Re: Entropy Explained, with Sheep (2016)

#14

Post by xtrapol8 »

kelnos wrote: post #13
IncreasePosts wrote: post #6

Earlier quoted context omitted.

Consider that if you Google for `"resistive variance" entropy`, the only hit is for the hn comment. It doesn't make sense because what they wrote makes no sense. Probably some looney with their own definition of entropy.

Your post is a good primer on bullshit detection: if you read something on the internet that sounds confidently authoritative, but you yourself are not well-versed in the subject, find what seems to be some key phrases and search the web for them. If you find lots of other hits on seemingly-reputable sources, then what you've read may be correct. If you only find the thing that you've just read, it's bullshit, with h…

Entropy is not the number of possible states in a system. Entropy includes outcomes not predicted by the mental model. Convention teaches this incorrectly. I’m the black sheep, bhaaaa.

Re: Entropy Explained, with Sheep (2016)

#15

Post by xtrapol8 »

IncreasePosts wrote: post #12
xtrapol8 wrote: post #7

Earlier quoted context omitted.

Well, this looney likes to point out that the manifold surface area (which is not always uniform) determines the rate of or density of distribution. All this is accounted for in the math selected for whichever for instance (“number of possible states” would count them) though a superior definition is one which is the most general technically correct without enumerating exceptions or extraneous clauses. Resistance con…

I'm not interested in reading your word salad.

Then neg it like anything else you refuse to believe.

Re: Entropy Explained, with Sheep (2016)

#16

Post by xtrapol8 »

ninetyninenine wrote: post #5
xtrapol8 wrote: post #2

> entropy is just a fancy word for ‘number of possible arrangements’ It isn’t though. Entropy is a fancy word for potential distribution over negative potential. Negative potential is the “surface area” over which potential may distribute. The “number of possible arrangements” casually fits into this, yet misses some unintuitive possibilities, like the resistive variance or other characteristics not preempted by who…

>Entropy is a fancy word for potential distribution over negative potential. Negative potential is the “surface area” over which potential may distribute. I don't understand this. Please elucidate.

The inverse square law is an example of potential distribution, not number of states.

Re: Entropy Explained, with Sheep (2016)

#17

Post by LaundroMat »

So if I get this right, there is an infinitely small possibility that a cracked egg returns to its initial state. Imagine that happening and being put on video. We'd all believe we're living in a simulation and witnessed a glitch.

No-one would believe the scientists explaining that although highly improbable, the uncracked egg does make scientific sense.

Re: Entropy Explained, with Sheep (2016)

#18

Post by Maro »

I've written a few articles about Entropy (I'm a physicist working in DS).

Almost all of them have Python code to illustrate concepts.

-

1. Entropy of a fair coin toss - https://bytepawn.com/what-is-the-entropy-of-a-fair-coin-toss...

2. Cross entropy, joint entropy, conditional entropy and relative entropy - https://bytepawn.com/cross-entropy-joint-entropy-conditional...

3. Entropy in Data Science - https://bytepawn.com/entropy-in-data-science.html

4. Entropy of a [monoatomic] ideal gas with coarse-graining - https://bytepawn.com/entropy-of-an-ideal-gas-with-coarse-gra...

5. All entropy related posts - https://bytepawn.com/tag/entropy.html

Re: Entropy Explained, with Sheep (2016)

#19

Post by speakeron »

LaundroMat wrote: post #17

So if I get this right, there is an infinitely small possibility that a cracked egg returns to its initial state. Imagine that happening and being put on video. We'd all believe we're living in a simulation and witnessed a glitch. No-one would believe the scientists explaining that although highly improbable, the uncracked egg does make scientific sense.

It's not really whether it makes scientific sense or not, it's just that it's so very highly improbable (really, really improbable) that other explanations make more sense: the video's a fake, it's mass hysteria, or even that we're living in a simulation.

Re: Entropy Explained, with Sheep (2016)

#20

Post by dr_dshiv »

I did a deep dive on entropy a couple years ago. I found the concept to be much harder to understand than I expected! Specifically, it was confusing to shift from the intuitive but wrong “entropy is disorder” to “entropy is about the number of possible microstates in a macrostate” (Boltzmann Entropy) https://en.wikipedia.org/wiki/Boltzmann%27s_entropy_formula

I was extra confused when I discovered that a spread out cloud of hydrogen is lower entropy than the same cloud gravitationally bound together in a star. So entropy isn’t just about “spreading out,” either.

I found that Legos provide a really nice example to illustrate entropy, so I’ll share that here.

Consider a big pile of Legos, the detritus of many past projects. Intuitively, a pile of Legos is high entropy because it is disordered—but if we are trying to move beyond order/disorder, we need to relate it to micro states and macro states.

Therefore, a pile of Legos is high entropy because if you randomly swap positions of the pieces it will all be the same macrostate—ie a big pile of Legos. Nevertheless, each of the Lego pieces is still in a very specific position— and if we could clearly snapshot all those positions, that would be the specific microstate. That means that the macrostate of the pile has an astronomical number of possible microstates — there are many ways to reorganize the pieces that still look like a pile.

On the other hand, consider a freshly built Lego Death Star. This is clearly low entropy. But to understand why in terms of microstates, it is because very few Legos can be swapped or moved without it not really being a Death Star anymore. The low entropy is because there are very few microstates (specific Lego positions) that correspond to the given macro state (being a Death Star).

This specific case helped me grok Boltzmann entropy. To extend it, consider a box with a small ice crystal in it: this has many fewer possible microstates than the same box filled with steam. In the steam, molecules can pretty much we swapped and moved anywhere and the macrostate is the same. With the crystal, if you start randomly swapping molecules to different microstates, it stops being an ice crystal quickly. So an ice crystal is low entropy.

Now, the definition of what counts as a macrostate is very important in this… but this comment is long enough and I still haven’t gotten to the gym…

Post reply on HN