What Is Entropy?
jasonfantl.com
What Is Entropy?
1–10 of 123 posts
Re: What Is Entropy?
#2Its the name for the information bits you don't have.
More elaborately, its the number bits needed to fully specify something which is known to be in some broad category of state but the exact details to calculate it are unknown.
Re: What Is Entropy?
#3Nowadays, it seems to be a buzzword to confuse people.
We IT folk should find another word for disorder that increases over time, specially when that disorder has human factors (number of contributors, number of users, etc). It clearly cannot be treated in the same way as in chemistry.
Re: What Is Entropy?
#4Nowadays, it seems to be a buzzword to confuse people. We IT folk should find another word for disorder that increases over time, specially when that disorder has human factors (number of contributors, number of users, etc). It clearly cannot be treated in the same way as in chemistry.
Maybe you're confused by entropy? It's pretty well established in different domains. There are multiple ways to look at the same phenomenon, because it's ubiquitous and generalized across systems. It comes down to information and uncertainty. The article in question does attempt to explain all of this if you read it.
Re: What Is Entropy?
#5Here is a good overview on Entropy [1]
Re: What Is Entropy?
#6Nowadays, it seems to be a buzzword to confuse people. We IT folk should find another word for disorder that increases over time, specially when that disorder has human factors (number of contributors, number of users, etc). It clearly cannot be treated in the same way as in chemistry.
Maybe you're confused by entropy? It's pretty well established in different domains. There are multiple ways to look at the same phenomenon, because it's ubiquitous and generalized across systems. It comes down to information and uncertainty. The article in question does attempt to explain all of this if you read it.
Maybe I am.
The part of thr article on information theory is more about mathematics than software. I don't deny there could be some generalization there.
The problem I see is that this could slip to measure human actions, which are also source of uncertainty, but although the words fit, in this particular case I think associating it with classical entropy does more harm than good.
https://en.m.wikipedia.org/wiki/Software_rot
Entropy as described in this article (software entropy), to me, does not fall under the same generalization. It is a looser use of the word. I used it myself several times, but now people are buzzwording entropy all around, and I think that looser use should be retracted to avoid thinking of humans as numbers or particles.
Re: What Is Entropy?
#7I love that the author clearly describes why saying entropy measures disorder is misleading.
Re: What Is Entropy?
#8One thing that helped me was the realization that, at least as used in the context of information theory, entropy is a property of an individual (typically the person receiving a message) and NOT purely of the system or message itself.
> entropy quantifies uncertainty
This sums it up. Uncertainty is the property of a person and not a system/message. That uncertainty is a function of both a person's model of a system/message and their prior observations.
You and I may have different entropies about the content of the same message. If we're calculating the entropy of dice rolls (where the outcome is the 'message'), and I know the dice are loaded but you don't, my entropy will be lower than yours.
Re: What Is Entropy?
#9As a software engineer, I learned what entropy was in computer science when I changed the way that a function was called which caused the system to run out of entropy in production and caused an outage. Heh.
Re: What Is Entropy?
#10One thing that helped me was the realization that, at least as used in the context of information theory, entropy is a property of an individual (typically the person receiving a message) and NOT purely of the system or message itself. > entropy quantifies uncertainty This sums it up. Uncertainty is the property of a person and not a system/message. That uncertainty is a function of both a person's model of a system/…
Not true. The uncertainty of the dice rolls is not controlled by you. It is the property of the loaded dice itself.
Here's a better way to put it. If I roll the dice infinite times. The uncertainty of the outcome of the dice will become evident in the distribution of the outcomes of the dice. Whether you or another person is certain or uncertain of this does not indicate anything.
Now when you realize this you'll start to think about this thing in probability called frequentists vs. bayesian and you'll realize that all entropy is, is a consequence of probability and that the philosophical debate in probability applies to entropy as well because they are one and the same.
I think the word "entropy" confuses people into thinking it's some other thing when really it's just probability at work.