Earlier quoted context omitted.
Laugh. I was hoping no one would ask since I don't have a great answer! We used "An introduction to Hilbert space" by Young. It's fine? There's stuff in there like Sturm-Liouville systems that I don't particularly use or care for. I also never use it as a reference. Here's a hodgepodge of other books related to functional analysis that I like more, but don't directly answer your question. I got a lot of benefit out o…
Thank you! What is that extra power that comes from considering Hilbert spaces as opposed to staying in good ol' R^n?
The reason that I mention Hilbert spaces is that they have more structure than a general function space, which makes working with them easier, but still general enough to be useful. Essentially, we get an inner product as well as the ability to enumerate an orthonormal basis, which makes it feel more like working with linear algebra.
Even in R^n, I believe strongly that it's important to code with these abstractions. First, it makes parallelizing the algorithms easier. If we treat a vector as simply an array of numbers and use the dot product, then the code requires a more significant rewrite when moving to multi-computer parallelism. If we treat them as a generic vector object and have an interface that works with inner products, addition, scaling, etc, then the same code can work either in serial or in parallel given two different implementations of the vectors and their operations.
In addition to the abstractions, the choice of inner product is important. If we have a linear operator in a Hilbert space and discretize it, we generally have to discretize three things: the operator, it's Hilbert adjoint, and the inner product. If we implement this blindly, they're not consistent after discretization. Meaning, the property that we should get is that = , but this probably isn't true if not done carefully. Generally, we can freely discretize two out of three of those operations and then the third one needs to be adapted for consistency. Maybe you want to choose the discretization for the operator and its adjoint, but this probably requires a non dot product for an inner product. Alternatively, maybe you're optimizing some problem and realize that some of your variables are out of scale, so it's converging slowly. You can certainly just rescale your variables with a diagonal scaling. However, you can also change your inner product, which changes the gradient, which also rescales the problem in a different way.
Mostly, that's to say that an inner product should be a choice that one freely and intentionally makes. Learning to work in Hilbert spaces forces us to become comfortable with this approach.