Viewing profile — gongy

The improvement is real! And unlike a lot of research, the code actually runs well. I can reproduce the results using Modal GPUs, leaving the code here: https://github.com/mirage-p…

You can have load balanced inference and also fine tune models on Modal (disclaimer: I wrote this guide) You can get a load balanced endpoint serving hundreds of tok/s per GPU usin…