What is an eigenvalue?

I think what the comment is missing is that the class of linear f(z)'s is much larger than you might expect. Partly because it just is, and partly because we like it that way.

In the land of analog signal processing: any combination of capacitors, inductors, and resistors [https://soundcertified.com/wp-content/uploads/2020/04/speake...] is linear.

In the land of math abstractions of signal processing: differentiation, integration, finite-impulse-response (FIR) filters, IIR filters, frequency-domain equalization, etc. All linear. Remember, linearity is with respect to the full time history, so f(z) = z(t) - z(t-1) + z(t-2) is still linear.

So we already know the eigenvectors of that whole arbitrary pile of componentry! Given any box of the above components, we can exactly characterize its response to any input -- for all time -- by knowing a list of that system's eigenvalues -- one for each of the already-known eigenvectors.

That's the "frequency response" -- the eigenvectors are sinusoids, and the frequency response is the eigenvalue corresponding to each eigenvector (sinusoid). And of course the Fourier transform takes you back and forth from the time domain to the eigen-domain.

We liked this analytical framework so much that when we could fabricate nonlinear devices (transistors) easily, we purposefully arranged things so these devices were only used in a linear part of their response curve. Hence, amplifiers: f(z) = 11 z.

And then the musicians introduced distortion and f*cked it all up -- our system isn't in a linear regime, our old eigenvectors are meaningless, and we can't predict what will come out. Pure chaos.

Let me try. Let’s say you have some operator F = crazyweirdblackbox. Like, it exponentiates and does a bunch more complicated things. You can apply this operator to some input function and transform it into some output function. The programming analogy would be some piece of code that works on a lambda and spits out another lambda. The finite-dimensional linear algebra equivalent is a matrix. You would ideally want F to be invertible (or undoable) for all inputs. Just in case.

If g(x) is an eigenfunction of F, then h(x) = F(g(x)) is actually just a rescaled of g(x): h(x) = ag(x), for some constant a. No matter how complicated and hard-to-compute F is, it boils down to just one number a, when acting on some special function g(x).

So what? This only applies to special g(x), and not any choice for g. Let’s say that I have some special function y(x) that isn’t necessarily g(x). But I have a whole bunch of eigenfunctions of F called g_k(x). If I knew that F was linear (kind of a prerequisite for computing eigenfunctions anyway), then I can decompose some output function y(x) into a weighted sum of g_k: y(x) = sum([w_k g_k(x) for k in range(infinity)]).

Some abuse of Python list comprehension there.

So instead of evaluating F, which might be very hard computationally or numerically, we can instead do a for-loop over potentially easier functions g_k. And store some weights w_k that essentially describe how F transformed an input function into y. Easy-peasy.

And maybe I don’t want to evaluate the entire sum. So I could choose to evaluate only the “important” terms in the sum to approximate y(x).

The entirety of signal processing, much of quantum mechanics, much of electromagnetism, and many other partial differential equations can fit into this framework. And we use PDEs to describe F when we really have no idea how to even to write it down. But we can still compute eigenfunctions and therefore make progress in evaluating F since we know the effect of F on certain special functions.

Does that help?

I like this explanation, which puts the concept of operator at center and has some nice computational relations, and is related to the physical world by the notion of an observable, with an eigenfunction being a special kind of operator-function relationship. At the end, there's a nice non-QM application (Fourier transforms).

source: PW Atkins & RS Friedman, "Molecular Quantum Mechanics 3rd Ed"

> An observable is any dynamical variable that can be measured... in classical mechanics... observables are represented by functions* (such as position as a function of time), in quantum mechanics they are represented by mathematical operators... We shall not in general distinguish between the observable and the operator that represents that observable (such as the position of a particle along the x-axis)"

> "An operator is a symbol for an instruction to carry out some action, an opeeation, on a function...in certain cases, the outcome of an operation is the same function, multiplied by a constant" [that constant being the eigenvalue]

Form: (operator) (eigenfunction) = (eigenvalue) (eigenfunction)

> "An important point is that a general function can be expanded in terms of all the eigenfunctions of an operator, a so-called complete set of functions... then a general function can be expressed as the linear combination [a sum over a complete set of functions, each function having its own coefficient]"

Finally, we get to a practical real-world example (non-QM):

> "...for instance, the straight line g = ax can be recreated over a certain range by superimposing an infinite number of sine functions, each of which is an eigenfunction of the [differentiation] operator, d2/dx2. Alternatively, the same function may be constructed from an infinite number of exponential functions, which are eigenfunctions of d/dx."

Extending this general concept, we can go into the famous Fourier Transform, used widely for all kinds of classical problems in converting waveforms into frequency peaks, such as in musical analysis, electrical engineering, etc. The wiki page on Fourier transforms has a very brief mention of this view, i.e. The Fourier transform decomposes a function into eigenfunctions for the group of translations.

Here's what looks like a deep dive into this approach to the Fourier Transform (2008):

http://www.systems.caltech.edu/dsp/ppv/papers/journal08post/...

I went through Linear Algebra I & II where eigenvalues & eigenvectors played a central role many a time and I was left with no elucidation regarding their practical applications.

Now I know that eigen*s (at risk of egregious oversimplification) can characterize systems and transformations, they are fundamentally widely applicable. Think of stretching narrow an arbitrary 2D image on a non-cardinal axis; an eigenvector can be a key factor that describes that transformation.

By far the best explanation of eigenvalues is found on 3blue1brown: https://www.youtube.com/watch?v=PFDu9oVAE-g&list=PLZHQObOWTQ...

Seriously, the course on the essence of linear algebra did more for me than my entire college linear algebra course. It's only a pity I discovered the videos at the end of my mechanical engineering degree and not sooner.