At the time of his death by a Roman soldier the ancient mathematician Archimedes is said to yell: Don't disturb my circles, while he was calculating on sand. Much later, a few years ago, one of his handbooks, an overwritten palimpsest, was found to contain elements of modern calculus. If both these concepts where saved and spread through the middle ages, human civilisation might have been developed 1000 years earlier.
The unreasonable effectiveness of the Fourier transform
121–130 of 167 posts
Re: The unreasonable effectiveness of the Fourier transform
#122Earlier quoted context omitted.
There is a saying about Gauss: when another mathematician came to show him a new result, Gauss would remark that he had already worked on it, open a drawer in his desk, and pull out a pile of papers on the same topic.
> There is a saying about Gauss: when another mathematician came to show him a new result, Gauss would remark that he had already worked on it, open a drawer in his desk, and pull out a pile of papers on the same topic. As if phd students need more imposter syndrom to deal with. Ona serious side, I wonder what conditions allow such minds to grow. I guess a big part is genetics, but I am curious if the "epi" is releva…
Imposter syndrome? If I was a PhD-level student (back then) and had an idea - and it turned out that Gauss had also thought of the idea, then written it out, and he kept the notes right in his desk - yeah. I'd take that as proof that I was one of the world's top mathematicians.
Re: The unreasonable effectiveness of the Fourier transform
#123People go all dopey eyed about "frequency space", that's a red herring. The take away should be that a problem centric coordinate system is enormously helpful. After all, what Copernicus showed is that the mind bogglingly complicated motion of planets become a whole lot simpler if you change the coordinate system. Ptolemaic model of epicycles were an adhoc form of Fourier analysis - decomposing periodic motions over…
All these transforms are switching to an eigenbasis of some differential operator (that usually corresponds to a differential equation of interest). Spherical harmonics, Bessel and Henkel functions, which are the radial versions of sines/cosines and complex exponential, respectively, and on and on. The next big jumps were to collections of functions not parameterized by subsets of R^n. Wavelets use a tree shapes para…
Note that I'm not great at math so it's possible I've entirely misunderstood you.
Here's an example of directly leveraging a transform to optimize the training process. ( https://arxiv.org/abs/2410.21265 )
And here are two examples that apply geometry to neural nets more generally. ( https://arxiv.org/abs/2506.13018 ) ( https://arxiv.org/abs/2309.16512 )
Re: The unreasonable effectiveness of the Fourier transform
#124Re: The unreasonable effectiveness of the Fourier transform
#125People go all dopey eyed about "frequency space", that's a red herring. The take away should be that a problem centric coordinate system is enormously helpful. After all, what Copernicus showed is that the mind bogglingly complicated motion of planets become a whole lot simpler if you change the coordinate system. Ptolemaic model of epicycles were an adhoc form of Fourier analysis - decomposing periodic motions over…
It’s kind of intriguing that predicting the future state of any quantum system becomes almost trivial—assuming you can diagonalize the Hamiltonian. But good luck with that in general. (In other words, a “simple” reference frame always exists via unitary conjugation, but finding it is very difficult.)
Re: The unreasonable effectiveness of the Fourier transform
#126My favorite story about the Fourier Transform is that Carl Friedrich Gauss stumbled upon the algorithm for the Fast Fourier Algorthim over a century before Cooley and Tukey’s publication in 1965 (which itself revolutionized digital signal processing).[1] He was apparently studying the motion of the asteroids Pallas and Juno and wrote the algorithm down in his notes but it never made it into public knowledge. [1] http…
When I interned at Chevron someone said they (or some other oil company) were using Fourier transforms in the 1950's for seismic analysis but kept it a secret for obvious reasons. I think you couldn't (can't?) patent math equations.
Re: The unreasonable effectiveness of the Fourier transform
#127Earlier quoted context omitted.
All these transforms are switching to an eigenbasis of some differential operator (that usually corresponds to a differential equation of interest). Spherical harmonics, Bessel and Henkel functions, which are the radial versions of sines/cosines and complex exponential, respectively, and on and on. The next big jumps were to collections of functions not parameterized by subsets of R^n. Wavelets use a tree shapes para…
Note that I'm not great at math so it's possible I've entirely misunderstood you. Here's an example of directly leveraging a transform to optimize the training process. ( https://arxiv.org/abs/2410.21265 ) And here are two examples that apply geometry to neural nets more generally. ( https://arxiv.org/abs/2506.13018 ) ( https://arxiv.org/abs/2309.16512 )
From the abstract and skimming a few sections of the first paper, imho it is not really the same. The paper is moving the loss gradient to the tangent dual space where weights reside for better performance in gradient descent, but as far as I understand neither the loss function nor the neural net are analyzed in a new way.
The Fourier and Wavelet transforms are different as they are self-adjoint operators (=> form an orthogonal basis) on the space of functions (and not on a finite dimensional vector space of weights that parametrize a net) that simplify some usually hard operators such as derivatives and integrals, by reducing them to multiplications and divisions or to a sparse algebra.
So in a certain sense these methods are looking at projections, which are unhelpful when thinking about NN weights since they are all mixed with each other in a very non-linear way.
Re: The unreasonable effectiveness of the Fourier transform
#128Earlier quoted context omitted.
> as he would simply end up in his father's shadow as he deemed it utterly Impossible to surpass his brilliance in maths Definitely true but also bad parenting. Gauss was somewhat of a freak of nature when it came to math. Him and Euler are two of the most unreasonably productive mathematicians of all time.
But what he deemed being posited as true, was this really bad parenting? It could be to head off competition or it could be brutal realism to head off future depression. Nepotism existed since time immemorial but for a mathematical genius, what was the nepotistic deliverable for the child? A sinecure placement at university?
> But what he deemed being posited as true
Implicit in the "correctness" of this motive is the idea that unless you're #1 in your field, you are nothing (depression implies strong feelings of worthlessness).
I don't know if you think that's a great lesson to teach your kids as a parent, but I don't.
Re: The unreasonable effectiveness of the Fourier transform
#129Earlier quoted context omitted.
It’s unusual to tell others not to do something because you’re projecting they’re secretly doing it to compete with you, or that they’ll be depressed when they don’t do what you did. Doubly so when the rationale is “I’m so fucking awesome” Triply so when it’s something you’re passionate about, presumably inherently. Quadruply so when it’s your child. Its tough as a kid hearing your parents come up with elongated excu…
I posit Gauss knew he was a GOAT and had ego. But I also posit he loved his children. So, a nepotistic delivery was beneficial for his family, and advising his son to seek excellence outside the shadow cast by Gauss himself wasn't stamping on dreams (in my view) it was seeking the happiest outcome. Without overdoing it, the suicide rate for rich kids with famous parents isn't nothing. There are positive examples, Ste…
What does "She isn't wings" mean?
Re: The unreasonable effectiveness of the Fourier transform
#130A signal cannot be both time and frequency band limited. Many years ago I was amazed when I read that this fact I learned in my undergraduate is equivalent to the Uncertainty Principle! On a more mundane note: my wife and I always argue whose method of loading the dishwasher is better: she goes slow and meticulously while I do it fast. It occurred to me we were optimizing for frequency and time domains, respectively,…
Another example: ears are excellent at breaking down the frequency of sounds, but are imprecise about where the sound is coming from; whereas eyes are excellent at telling you where light is coming from, but imprecise about how its frequencies break down.