PyTorch 2 Internals [slides]

As an end user, who cares? Make it more like Keras. Quit messing about w tensor optimization because you'll get smoked by someone else's performance eventually.

Last time I checked, there wasn't even a function to shuffle a tensor. Something you'd do at the start of every training iteration.