Viewing profile — jjssmith

> One really impressive thing about OpenAI is that most of the people running the company (on the technical side, at least) are under 35. Many of the important decisionmakers behin…

LOL. This is a classical technique, Johnson-Linderstrauss etc. In this context, rediscovered every few years (recently months), e.g. here's 2017: https://proceedings.mlr.press/v70/…

You might like an information-theoretic take on SpinQuant and the likes [1]. tl;dr: round((2*R)*x) is not a great idea for an R-bit quantization. [1] https://arxiv.org/abs/2410.137…

AFAIK, the Markov chain page ranker existed before Google (Jon Kleinberg's papers, e.g.), what Google paper added were some smaller bells and whistles, like using the text descript…

A big part (majority?) of this work is rediscovery of the results from info theory (EE) community in the 90s (ctrl-f Orlitsky, Shor in the pdf).

Rediscovering NLMS now, good for ML folks. https://en.wikipedia.org/wiki/Least_mean_squares_filter#Norm...