Viewing profile — gajjanag

> Communication matters most when you're dealing with cross-org concerns and those that master it are usually the more friendly and pleasant ones. I don't agree with the second one…

> Thanks for that! It is worth noting that taking advantage of the post-rotation distribution I again feel this claim is too strong. Rotations have been used in information theory/…

Wow, yes - you are completely correct (read through the note in detail now). Though, as your paper also notes, the quantizer values themselves aren't fundamentally novel to either …

There are also more papers on similar themes. For example, TurboQuant makes use of QJL (quantized Johnson Lindenstrauss transformations). One of the first papers to characterize th…

TurboQuant is known across the industry to not be state of the art. There are superior schemes for KV quant at every bitrate. Eg, SpectralQuant: https://github.com/Dynamis-Labs/spe…

The bigger challenge is GPU/NPU. Branches for fast vs accurate path get costlier, among other things. On CPU this is less of a cost. Most published libm on GPU/NPU side have a few …

> That's why you need to put your scope The problem is, "scope" is often equated to "how many people worked in my empire" rather than "how much business value did my work X generat…

>80%-90% or so of real life vectorization can be achieved in C or C++ just by writing code in a way that it can be autovectorized. Yep. I was pleasantly surprised by the autovector…

Welcome to the brave new world these days: 1 - Very few people conduct "proper scholarship", and fail to trace ideas back to their original inception and cite them correctly. This …

> I don't think there are many (or any) upsides to the well documented downsides. C++ template metaprogramming still remains extremely powerful. Projects like CUTLASS, etc could no…

As others have pointed out, these phenomena are well known to many folks across companies in the AI infra space. It doesn't really break new ground. This article is a good expositi…

I guess you have never worked with a slow induction cooktop. Literally we had to spend 15 minutes more for cooking things on induction compared with our previous apartment's gas co…

+1 - there are just so many Asian recipes that can not be done anywhere near as easily on induction stovetops (high heat from direct flame for flatbreads, etc). Plus a whole bunch …

There is a vast number of sysctl in xnu that have not really been re-examined in over 15 years. Many tunings date back to the spinning rust drive era (for example). There are plent…

The big problem is a bunch of folks actually take these things seriously and use it as an excuse to freeze the junior hiring pipeline. At the senior levels this is not actually bel…

> Large corporations believe anyone is replaceable. This is definitely true. By design, large corporations are structured so that there is no single point of failure. > Again I am …

Same. The compensation is substantially better at FAANG, but in terms of actual on the ground work being rewarded, almost never the case. Meta-work (lots of "cross functional" docu…

This is much more nuanced now. See Apple "Private Cloud Compute": https://security.apple.com/blog/private-cloud-compute/ ; they run a lot of the larger models on their own servers.…

Maybe on a particular model/dataset but extremely unlikely in general. Again, like another commenter pointed out: if you truly believe it isn't that hard we would love to hire you …

Our group works on some of this stuff at Meta, and we have a pretty good diversity of backgrounds - high performance computing (the bulk), computer systems, compilers, ML engineers…

lmkd (low memory killer daemon) works fairly differently off of a different set of signals and different policy. But yes, conceptually they try to achieve the same goal. I also do …

A couple of additional points on how the "low-RAM" works: 1 - https://www.lifewire.com/understanding-compressed-memory-os-... : Apple devices have support for memory compression, s…

Umm, Apple still sells devices with just 1 GB of RAM on them ;)

Same, doing it once out of my own curiosity to see how the corporate machine works. Not doing it again - seeing first hand how it is due to managerial incompetence more than anythi…

> based on "impact" rather than arbitrary metrics Umm, from whatever I have seen in big tech "impact" is also fairly arbitrary. It all is based on how cozy one is with one's manage…