In my layman’s naive view the promise of generative AI is the ability to estimate highly dimensional non linear systems effectively and efficiently. At some level these can be viewed as solving non linear systems. In my career a specific class of important problems has been estimating systems of partial different equations, and specially stochastic partial differential equations. Monte Carlo methods are often the mos…
One way to think about these methods is that we are essentially implementing a Monte-Carlo algorithm physically, where on each "iteration" there is a matrix-vector multiplication. The physical system does this matrix-vector multiplication for us in constant time, so it does have an advantage over these digital methods. Not only that, but the "clock speed" of the physical system can be almost arbitrarily short, although this comes with an energy cost.