Viewing profile — kbwal7

I'm honestly not too aware of the true scene of which optimizers are used nowadays by which labs (though I think Moonshot has shown Muon's ability to scale to large models, and Muo…

I do think there are now open weight models that are on par with (or beating) Opus 4.5 by now (e.g. Kimi K3, GLM5.3). But yeah obviously the frontier closed source models seem to h…

Olmo is one truly open source model. https://allenai.org/blog/olmo3

I think it's just a harness that uses proprietary (or maybe IBM is hosting open weight?) models. They say on the site (in the FAQs) that you can't even choose which model! So it se…

It's pretty incredible how robust neural networks are to even things like architectural changes mid-run. The team at Meta when training their own copy of GPT-3 (called OPT-3) even …