Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is entropy? A measure of just how little we know

quantamagazine.org

Post reply on HN
Print view View original

71–80 of 169 posts

Re: What is entropy? A measure of just how little we know

#71

Post by oh_my_goodness »

kergonath wrote: post #64
LudwigNagasena wrote: post #40

Earlier quoted context omitted.

> It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. Articles about MaxEnt thermodynamics by E.T. Jaynes where he talks about the “anthropomorphic” nature of entropy date back to 1960s. How is that not right in physics?

By the look of it, it is another misguided attempt to apply information theory concepts to thermodynamics. Entropy as information is seductive because that way we think we can understand it better, and it looks like it works. But we need to be careful because even though we can get useful insights from it (like Hawking radiation) it’s easy to reach unphysical conclusions. > How is that not right in physics? Why would…

"We need to be careful", agreed. "It's not mundane", agreed. (It's mundane in information theory because that's how they define entropy.)

"It's [...] not right" (from your first comment), can you give/link a specific physical example? It would be very cool to have a clear counterexample.

Re: What is entropy? A measure of just how little we know

#72

Post by azemetre »

zV62drdTw6CM wrote: post #68

Happy to see the article being discussed here! I was responsible for the technical implementation of the interactives. If you are interested in the source code, you can find it here: https://github.com/jnsprnw/mip-entropy It’s built in Svelte 5 with Tailwind.

May I ask why you chose svelte 5 rather than something else? I’ve noticed that a lot of “one off” interactions are being built with svelte nowadays. What are the benefits of using it?

Re: What is entropy? A measure of just how little we know

#73

Post by [deleted] »

markisus wrote: post #49
kergonath wrote: post #31

Earlier quoted context omitted.

> Maybe I have the benefit of giant shoulders, but this seems like a fairly mundane observation. It is not mundane, and it is also not right, at least for entropy in Physics and Thermodynamics. > High-entropy states are those macrostates which have many corresponding microstates. That is how you deduce entropy form a given model. But entropy is also something that we can get from experimental measurements. In this ca…

Maybe the experimental apparatus is not objective. The quantities we choose to measure are dictated by our psychological and physiological limitations. The volume of a container is not objectively defined. An organism which lives at a faster time scale will see the walls of the container vibrating and oscillating. You must convince the organism to average these measurements over a certain time scale. This averaging t…

[deleted]

Re: What is entropy? A measure of just how little we know

#74

Post by mitthrowaway2 »

LudwigNagasena wrote: post #38
guerrilla wrote: post #8

Earlier quoted context omitted.

> mundane observation What makes an observation mundane? I think what you said is insightful and demonstrates intelligence. I don't think it's at all obvious to the masses of students that have poured over introductory physics textbooks. In fact, it seems to me that often entropy is taught poorly and that very few people understood it well but that we are beginning to correct that. I point to the heaps of popsci maga…

Maybe if you opened only introductory physics textbooks then it’s not mundane; but If you opened introductory information theory textbooks, statistical textbooks, introductions to Bayesian probability theory, articles about MaxEnt thermodynamics including articles by E.T. Jaynes, then it’s a quite mundane observation.

Jaynes definitely made the case for this a long time ago, and in my opinion he's correct, but I think that view is still not mainstream; or even if mainstream, certainly not dominant. So I think we should welcome other people who reach it on their own journey, even if they aren't the first to arrive there.

Re: What is entropy? A measure of just how little we know

#75

Post by mitthrowaway2 »

philipov wrote: post #55
adrianN wrote: post #53

Earlier quoted context omitted.

Is that true? It was my understanding that chaotic systems (with a known initial state) could be predicted to arbitrary precision by throwing enough compute at the problem. Of course knowing the initial state is impossible in the real world, „enough“ compute might not fit in the observable universe, and quantum mechanics contains true randomness…

Yes, chaotic systems can be simulated, and that's the only way to find out what happens with them. The thing one can't do is reduce them to a closed-form function of x and compute their value for any point along their domain. The only thing we can do is inductively figure out the next x from the current x .

The issue is more that simulations diverge exponentially with time from any nonzero error in the initial state. With perfect knowledge of the initial conditions and parameters you could simulate the system perfectly, but you'd also need to accumulate zero numerical error along the way.

If you don't demand perfection, then in practice you can do pretty well for short times.

Re: What is entropy? A measure of just how little we know

#76

Post by zV62drdTw6CM »

azemetre wrote: post #72
zV62drdTw6CM wrote: post #68

Happy to see the article being discussed here! I was responsible for the technical implementation of the interactives. If you are interested in the source code, you can find it here: https://github.com/jnsprnw/mip-entropy It’s built in Svelte 5 with Tailwind.

May I ask why you chose svelte 5 rather than something else? I’ve noticed that a lot of “one off” interactions are being built with svelte nowadays. What are the benefits of using it?

First, when I joined the project, it wasn’t clear how it would be published. Svelte, which outputs compiled JavaScript, can fit into many CMS workflows that newspapers/publishers use.

I believe Svelte was developed by Rich Harris at the NY Times for this very reason.

We ended up using iFrames, so other frameworks like React could have been used.

Second, Svelte is very well suited for these small interactives because it has built-in state, transitions, and reactivity with low overhead.

Third, it was a personal choice, as I now do most of my work in Svelte.

Re: What is entropy? A measure of just how little we know

#77

Post by notfed »

kergonath wrote: post #61
markisus wrote: post #49

Earlier quoted context omitted.

Maybe the experimental apparatus is not objective. The quantities we choose to measure are dictated by our psychological and physiological limitations. The volume of a container is not objectively defined. An organism which lives at a faster time scale will see the walls of the container vibrating and oscillating. You must convince the organism to average these measurements over a certain time scale. This averaging t…

> The quantities we choose to measure are dictated by our psychological and physiological limitations. No. The enthalpy changes measured by a calorimeter are not dependent on our psychological limitations. > The volume of a container is not objectively defined. Yes, it is, for any reasonable definition of "objective". We know how to measure lengths, we know how they change when we use different frames of reference so…

Sometimes I wish HN had merit badges. Or if you like, a device to measure the amount of information contained within a post.

Re: What is entropy? A measure of just how little we know

#78

Post by kqr »

zV62drdTw6CM wrote: post #76
azemetre wrote: post #72

Earlier quoted context omitted.

May I ask why you chose svelte 5 rather than something else? I’ve noticed that a lot of “one off” interactions are being built with svelte nowadays. What are the benefits of using it?

First, when I joined the project, it wasn’t clear how it would be published. Svelte, which outputs compiled JavaScript, can fit into many CMS workflows that newspapers/publishers use. I believe Svelte was developed by Rich Harris at the NY Times for this very reason. We ended up using iFrames, so other frameworks like React could have been used. Second, Svelte is very well suited for these small interactives because…

> We ended up using iFrames, so other frameworks like React could have been used.

Wait, wasn't one of the original selling points of React that it could be embedded piecewise to enhance interactivity of the parts of pages that needed it? It should certainly not need a separate page!

Re: What is entropy? A measure of just how little we know

#79

Post by whatshisface »

kergonath wrote: post #61
markisus wrote: post #49

Earlier quoted context omitted.

Maybe the experimental apparatus is not objective. The quantities we choose to measure are dictated by our psychological and physiological limitations. The volume of a container is not objectively defined. An organism which lives at a faster time scale will see the walls of the container vibrating and oscillating. You must convince the organism to average these measurements over a certain time scale. This averaging t…

> The quantities we choose to measure are dictated by our psychological and physiological limitations. No. The enthalpy changes measured by a calorimeter are not dependent on our psychological limitations. > The volume of a container is not objectively defined. Yes, it is, for any reasonable definition of "objective". We know how to measure lengths, we know how they change when we use different frames of reference so…

The enthalpy changes measured by a calorimeter are dependent on the design of the calorimeter, which could have been a different piece of equipment. In a sense, that makes it dependent on the definition of enthalpy.

If you introduced a new bit of macro information to the definition of an ensemble, you'd divide the number of microstates by some factor. That's the micro level equivalent of macroscopic entropy being undefined up to an additive constant.

The measurables don't tell you S, they only tell you dS.

Re: What is entropy? A measure of just how little we know

#80

Post by currymj »

tsoukase wrote: post #13

My perspective: 1) entropy is a mathematical concept. Physics apply it, like any other mathematical one 2) math entropy does not change, everything is reversible, as it is based on known quantities 3) entropy measures how far we are from perfectly knowing a system 4) logarithm is chosen to measure entropy because of its convenient properties, not because of any deeper law. It could be a sum, a product etc 5) the Seco…

i think those convenient properties are the sign of some deeper law. you do want to work with the exponential and logarithm, same as in so many other places. you don’t necessarily want some other generalized “entropy”.
Post reply on HN