1M+ LoC, 400 models still afloat: The engineering behind Transformers

interesting blog if not just for the interactive plots: getting to see how all the common ML architectures are linked together and their evolution has value.

I'm guessing that as we go deeper, we'll see fewer and fewer completely novel architectures that don't depend on the previous ones.