Live data from Hacker News

An alternative construction of Shannon entropy

rkp.science

11–12 of 12 posts

Re: An alternative construction of Shannon entropy

#11

Calling this 'alternative' construction seems like coming full circle since this line of combinatorial argument is how Boltzmann came up with his H-function in the first place, which inspired Shannon's entropy.

Yep! This relationship is well known in statistical mechanics. I was just surprised that in many years of intersecting with information theory in other fields (computational neuroscience in particular) I'd never come across it before, even though IMO it provides an insightful perspective.
Post reply on HN