Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The unreasonable effectiveness of the Fourier transform

joshuawise.com

Post reply on HN
Print view View original

161–167 of 167 posts

Re: The unreasonable effectiveness of the Fourier transform

#161

Post by tsoukase »

hahahahhaah wrote: post #114
btilly wrote: post #93

Earlier quoted context omitted.

For those who don't get this comment, the Heisenberg uncertainty principle applies to any two quantities that are connected in QM via a Fourier transform. Such as position and momentum, or time and energy. It is really a mathematical theorem that there is a lower bound on the variance of a function times the variance of its Fourier transform. That lower bound is the uncertainty principle, and that lower bound is hit…

Thats. I always assumed it was more a quirk of the universe than something driven by pure mathematics. Amazing.

I feel the same holds for the 2nd thermodynamic law. It's mathematically imposed that the most probable event will happen eventually, so a tautology. It's not that some gas or molecules cannot be reverted to a previous situation.

Re: The unreasonable effectiveness of the Fourier transform

#162

Post by tsoukase »

hahahahhaah wrote: post #67
Jun8 wrote: post #58

A signal cannot be both time and frequency band limited. Many years ago I was amazed when I read that this fact I learned in my undergraduate is equivalent to the Uncertainty Principle! On a more mundane note: my wife and I always argue whose method of loading the dishwasher is better: she goes slow and meticulously while I do it fast. It occurred to me we were optimizing for frequency and time domains, respectively,…

The self loading dishwasher would be the greatest marriage saving invention since car navigation systems.

One of the things that I feel blessed of in my marriage is that my wife has the same way of loading the dishwasher like me. Anyway, fellow husbands think about the dozens of other conflicts that you are might avoiding.

Re: The unreasonable effectiveness of the Fourier transform

#164

Post by srean »

madhadron wrote: post #155
srean wrote: post #134

Earlier quoted context omitted.

You may well be right about neural networks. Sometimes models that seem nonlinear turns linear if those nonlinearities are pushed into the basis functions, so one can still hope. For GPT like models, I see sentences as trajectories in the embedded space. These trajectories look quite complicated and no obvious from their geometrical stand point. My hope is that if we get the coordinate system right, we may see someth…

I’m not sure what you mean by a change of basis making a nonlinear system linear. A linear system is one where solutions add as elements of a vector space. That’s true no matter what basis you express it in.

It depends on parameterization.

For example, if you prameterize the x,y coordinates of a plane-circular trajectory in terms the angle theta, it's nonlinear function of theta.

However, if you parameterized a point in terms of the tuple (cos \theta, sin \theta) it comes out as a scaled sum. Here we have pushed the nonlinear functions cos and sin inside the basis functions.

A conic section is nonlinear curve (not a line) when considered in the variables of and y. However, in the basis of x^2, xy, y^2, x, y it's linear (well, technically affine).

Consider the Naive Bayes classifier. It looks nonlinear till one parameterized it in log p, then it's linear in log-p and log-odds.

If one is ok with dimensional basis this linearisation idea can be pushed much further. Take a look at this if you are interested

https://math.stackexchange.com/questions/4471490/a-proper-ap...

Re: The unreasonable effectiveness of the Fourier transform

#165

Post by srean »

anamax wrote: post #158
srean wrote: post #113

People go all dopey eyed about "frequency space", that's a red herring. The take away should be that a problem centric coordinate system is enormously helpful. After all, what Copernicus showed is that the mind bogglingly complicated motion of planets become a whole lot simpler if you change the coordinate system. Ptolemaic model of epicycles were an adhoc form of Fourier analysis - decomposing periodic motions over…

> My suspicion is that we are in Ptolemaic state as far as GPT like models are concerned. We will eventually understand them better once we figure out what's the better coordinate system to think about their dynamics in. Most deep learning systems are learned matrices that are multiplied by "problem-instance" data matrices to produce a prediction matrix. The time to do said matrix-multiplication is data-independent (…

Neural nets are quite a bit more than matrix multiplications, at least in their current representation.

There are layers upon layers of nonlinearity, be it with softmax or sigmoid. In the tangent kernel view it does linearize.

Re: The unreasonable effectiveness of the Fourier transform

#166

Post by srean »

tsoukase wrote: post #121

At the time of his death by a Roman soldier the ancient mathematician Archimedes is said to yell: Don't disturb my circles, while he was calculating on sand. Much later, a few years ago, one of his handbooks, an overwritten palimpsest, was found to contain elements of modern calculus. If both these concepts where saved and spread through the middle ages, human civilisation might have been developed 1000 years earlier…

Yes! Archimedes was probably alien intelligence compared to his peers.

His method of exhaustion does qualify as integral calculus and it's not limited to that overwritten document. It appears many times in his work

Re: The unreasonable effectiveness of the Fourier transform

#167

Post by cbracketdash »

alberto_ol wrote: post #136
Qem wrote: post #103

Earlier quoted context omitted.

Why Johnny can't be unreasonably effective?

What we talk about when we talk about the unreasonable effectiveness

Reasonable effectiveness: an alternative to unreasonable effectiveness
Post reply on HN