Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unreasonable effectiveness of the Fourier transform

joshuawise.com

Post reply on HN
Print view View original

111–120 of 167 posts

Re: The unreasonable effectiveness of the Fourier transform

#111

Post by srean »

groby_b wrote: post #15
redhed wrote: post #8

Earlier quoted context omitted.

It is pretty funny to flippantly call an influential paper by someone who received a Nobel Prize in Physics 'asinine'.

I mean... this one's actually a pretty good paper, but we also had Linus Pauling pontificate on Vitamin C, so maybe we should cool it with the appeals to Nobel authority alone.

He did have a very long life, so there's that.

It's not easy to separate cause and effect from direct and strong correlations that we experience.

The job of a scientist is not to give up on a hunch with a flippant "correlation is not causation" but pursue such hunches to prove it this way or that (that is, prove it or disprove it). It's human to lean a certain way about what could be true.

Re: The unreasonable effectiveness of the Fourier transform

#113

Post by srean »

People go all dopey eyed about "frequency space", that's a red herring. The take away should be that a problem centric coordinate system is enormously helpful.

After all, what Copernicus showed is that the mind bogglingly complicated motion of planets become a whole lot simpler if you change the coordinate system.

Ptolemaic model of epicycles were an adhoc form of Fourier analysis - decomposing periodic motions over circles over circles.

Back to frequencies, there is nothing obviously frequency like in real space Laplace transforms *. The real insight is that differentiation and integration operations become simple if the coordinates used are exponential functions because exponential functions remain (scaled) exponential when passed through such operations.

For digital signals what helps is Walsh-Hadamard basis. They are not like frequencies. They are not at all like the square wave analogue of sinusoidal waves. People call them sequency space as a well justified pun.

My suspicion is that we are in Ptolemaic state as far as GPT like models are concerned. We will eventually understand them better once we figure out what's the better coordinate system to think about their dynamics in.

* There is a connection though, through the exponential form of complex numbers, or more prosaically, when multiplying rotation matrices the angles combine additively. So angles and logarithms have a certain unity, or character.

Re: The unreasonable effectiveness of the Fourier transform

#114

Post by hahahahhaah »

btilly wrote: post #93
hammock wrote: post #83

Earlier quoted context omitted.

It’s literally the Heisenberg uncertainty principle, applied to signal processing.

For those who don't get this comment, the Heisenberg uncertainty principle applies to any two quantities that are connected in QM via a Fourier transform. Such as position and momentum, or time and energy. It is really a mathematical theorem that there is a lower bound on the variance of a function times the variance of its Fourier transform. That lower bound is the uncertainty principle, and that lower bound is hit…

Thats. I always assumed it was more a quirk of the universe than something driven by pure mathematics. Amazing.

Re: The unreasonable effectiveness of the Fourier transform

#115

Post by alexlesuper »

srean wrote: post #113

People go all dopey eyed about "frequency space", that's a red herring. The take away should be that a problem centric coordinate system is enormously helpful. After all, what Copernicus showed is that the mind bogglingly complicated motion of planets become a whole lot simpler if you change the coordinate system. Ptolemaic model of epicycles were an adhoc form of Fourier analysis - decomposing periodic motions over…

I feel like this is the way we should have learned Fourier and Laplace transforms in my DSP class. Not just blindly applying formulas and equations.

Re: The unreasonable effectiveness of the Fourier transform

#116

Post by Gannon6790 »

btilly wrote: post #93
hammock wrote: post #83

Earlier quoted context omitted.

It’s literally the Heisenberg uncertainty principle, applied to signal processing.

For those who don't get this comment, the Heisenberg uncertainty principle applies to any two quantities that are connected in QM via a Fourier transform. Such as position and momentum, or time and energy. It is really a mathematical theorem that there is a lower bound on the variance of a function times the variance of its Fourier transform. That lower bound is the uncertainty principle, and that lower bound is hit…

thank you for that reminder/clarification. I forget sometimes how much we think we have clear pictures of how things like that work when really we're just listening to someone trying to explain what the math is doing and we're adding in detail.

Re: The unreasonable effectiveness of the Fourier transform

#117

Post by skinkestek »

brcmthrowaway wrote: post #31
derektank wrote: post #23

My favorite story about the Fourier Transform is that Carl Friedrich Gauss stumbled upon the algorithm for the Fast Fourier Algorthim over a century before Cooley and Tukey’s publication in 1965 (which itself revolutionized digital signal processing).[1] He was apparently studying the motion of the asteroids Pallas and Juno and wrote the algorithm down in his notes but it never made it into public knowledge. [1] http…

How was Gauss so productive with 6 children?

I only have 5 kids, and I am also not nearly as productive as Gauss but to a certain degree, it feels to me like responsibility kind of tries to force me to be more effective.

Re: The unreasonable effectiveness of the Fourier transform

#118

Post by patentatt »

alexlesuper wrote: post #115
srean wrote: post #113

People go all dopey eyed about "frequency space", that's a red herring. The take away should be that a problem centric coordinate system is enormously helpful. After all, what Copernicus showed is that the mind bogglingly complicated motion of planets become a whole lot simpler if you change the coordinate system. Ptolemaic model of epicycles were an adhoc form of Fourier analysis - decomposing periodic motions over…

I feel like this is the way we should have learned Fourier and Laplace transforms in my DSP class. Not just blindly applying formulas and equations.

I’d argue that most if not all of the math that I learned in school could be distilled down to analyzing problems in the correct coordinate system or domain! The actual manipulation isn’t that esoteric once you get in the right paradigm. And those professors never explained things at that kind of higher theoretical level, all I remember was the nitty gritty of implementation. What a shame. I’m sure there’s higher levels of mathematics that go beyond my simplistic understanding, but I’d argue it’s enough to get one through the full sequence of undergraduate level (electrical) engineering, physics, and calculus.

Re: The unreasonable effectiveness of the Fourier transform

#119

Post by hammock »

hahahahhaah wrote: post #114
btilly wrote: post #93

Earlier quoted context omitted.

For those who don't get this comment, the Heisenberg uncertainty principle applies to any two quantities that are connected in QM via a Fourier transform. Such as position and momentum, or time and energy. It is really a mathematical theorem that there is a lower bound on the variance of a function times the variance of its Fourier transform. That lower bound is the uncertainty principle, and that lower bound is hit…

Thats. I always assumed it was more a quirk of the universe than something driven by pure mathematics. Amazing.

Yes that’s fair to say. The tradeoff is mathematically inevitable. Physics just dictates the constants.

It’s also the kind of thinking that can throw a wet blanket on the “beauty” of e.g. Eulers identity (not being critical, I genuinely appreciate the replies I got)

Re: The unreasonable effectiveness of the Fourier transform

#120

Post by madhadron »

srean wrote: post #113

People go all dopey eyed about "frequency space", that's a red herring. The take away should be that a problem centric coordinate system is enormously helpful. After all, what Copernicus showed is that the mind bogglingly complicated motion of planets become a whole lot simpler if you change the coordinate system. Ptolemaic model of epicycles were an adhoc form of Fourier analysis - decomposing periodic motions over…

All these transforms are switching to an eigenbasis of some differential operator (that usually corresponds to a differential equation of interest). Spherical harmonics, Bessel and Henkel functions, which are the radial versions of sines/cosines and complex exponential, respectively, and on and on.

The next big jumps were to collections of functions not parameterized by subsets of R^n. Wavelets use a tree shapes parameter space.

There’s a whole, interesting area of overcomplete basis sets that I have been meaning to look into where you give up your basis functions being orthogonal and all those nice properties in exchange for having multiple options for adapting better to different signal characteristics.

I don’t think these transforms are going to be relevant to understanding neural nets, though. They are, by their nature, doing something with nonlinear structures in high dimensions which are not smoothly extended across their domain, which is the opposite problem all our current approaches to functional analysis deal with.

Post reply on HN