Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Thermodynamic Linear Algebra

arxiv.org

Post reply on HN
Print view View original

11–20 of 60 posts

Re: Thermodynamic Linear Algebra

#12

Post by aifer4 »

foota wrote: post #11

Isn't there a relation between entropy and computation? It would be interesting to see how these are related.

Definitely. Landauer's principle gives a lower bound on the amount of energy a computation requires, which is k_B T ln(2) times the number of bits erased in the process, the decrease in Shannon entropy. Our energy cost analysis is not based on the Landauer limit, but simply on the energy difference between equilibrium states. But our algorithm for estimating the determinant is based on effectively measuring the entropy difference between equilibrium states.

Re: Thermodynamic Linear Algebra

#17

Post by aifer4 »

notphya wrote: post #15

We have mass-energy equivalence. Do we also have information-energy equivalence? Can we use the Landauer bounds to prove by transitivity that information and energy are equivalent?

Straying a bit off topic, but I think one of the more sensible approaches to information-energy equivalence is a thermodynamic engine with an information reservoir (in addition to the heat and work reservoirs normally considered). https://arxiv.org/pdf/1408.1224.pdf https://journals.aps.org/prx/pdf/10.1103/PhysRevX.3.041003

Re: Thermodynamic Linear Algebra

#18

Post by fnordpiglet »

In my layman’s naive view the promise of generative AI is the ability to estimate highly dimensional non linear systems effectively and efficiently. At some level these can be viewed as solving non linear systems. In my career a specific class of important problems has been estimating systems of partial different equations, and specially stochastic partial differential equations. Monte Carlo methods are often the most computable estimations. I’ve often found we approach these things by either actually linearizing or estimating a linearization of the system and using linear algebra to solve, then transforming back. All these techniques requires enormous amount of computation and extremely complex math and numerical methods. To my naive understanding (I’m more of a core systems person that’s been adjacent to the work) quantum promises to help here by directly simulating the system.

Would this thermodynamic technique provide solutions to these sorts of non linear optimization and system solving problems? It feels from my reading it might, and in a simpler way to express.

Forgive my likely display of extraordinary ignorance.

Re: Thermodynamic Linear Algebra

#19

Post by Talinx »

Does this hold up when taking quantum mechanics into account?

Let's assume you need at least m = n^2 particles for a physical system modelling a n by n matrix and model the change of the system from setting the state of the particles (to the matrix elements) to measurement by a finite number of interactions between particles (by exchanging a photon):

- a particle can interact with a particle of the heat bath

- a particle can interact with another particle of the m particles of the system

I guess this result holds up if the second interaction kind does not matter because the first interaction alone then takes a constant time for each particle. The whole thing becomes a massively parallel computation (with m threads).

But the second interaction should matter, otherwise how can the system capture/model dependencies between variables (I guess)?

My intuition would be that subsystems of particles get closer to the equilibrium by interaction with the heat bath and then two subsystems combine their wave functions to one by the second kind of interaction. You got subsystems that are in local thermal equilibrium that combine and split their wave functions and as time goes to t_0 the subsystems sizes that are in local equilibrium get larger and larger until they reach size m at time t_0. This does seem to take longer for more particles (not that massively parallel anymore). Anyone got any insight into how this scales?

(This only matters under the assumption that the number of photon exchanges (that each particle experiences) for each of the m particles is finite and constant (or gets larger with larger m) for a fixed temperature. I could easily have missed some things that could make these thoughts irrelevant.)

Re: Thermodynamic Linear Algebra

#20

Post by aifer4 »

Talinx wrote: post #19

Does this hold up when taking quantum mechanics into account? Let's assume you need at least m = n ^2 particles for a physical system modelling a n by n matrix and model the change of the system from setting the state of the particles (to the matrix elements) to measurement by a finite number of interactions between particles (by exchanging a photon): - a particle can interact with a particle of the heat bath - a par…

These results probably would not hold in the same form for a quantum system. By a quantum system, I mean a system where the decoherence time is on the order of the other timescales present in the system (e.g. the correlation time). In fact, it would be much more difficult to engineer such a system, and we would not want one for this purpose; the results rely on convergence to a classical canonical equilibrium distribution, which has to be generalized in the quantum case, meaning it may not have the properties we want. Also, we would have to deal with the measurement backaction on the system in the quantum limit, which we definitely don't want. In the classical limit, where the energy is much larger than Planck's constant divided by the timescale of the system, this is not an issue. One more thing: our algorithms use continuous measurement of the system. For a quantum system, due to the quantum Zeno effect, the system would be effectively "frozen", so we would definitely not sample the full distribution.
Post reply on HN