271–273 of 273 posts
Earlier quoted context omitted.
>pedantry I wasted 2 minutes of my life looking for the exponential reduction. So did many others. Now I'm wasting more of my life shit posting about it, but at least that's a conscious choice.
No, you spent 2 minutes improving your reading comprehension.
Earlier quoted context omitted.
The GEMM is O(n^3) actually. Transformers are quadratic in the size of their context window.
I read that as a typo given the next sentence. I’m on the fence about cubic time. I was mostly thinking of exponential and factorial problems. I think some very clever people can make cubic work despite my warnings. But most of us shouldn’t. General advice is to be ignored by masters when appropriate. That’s also the story arc of about half of kung fu movies. Did chess solvers really progress much before there was a…