Viewing profile — addiefoote8

I'm also excited about the research that could be enabled by having weight-level access and fine tuning access on frontier open source models. There's a lot of interesting behavior…

Some more details on this: After realizing Hugging Face would be messy to work with to train Kimi-k2-thinking, we decided to do it ourselves. We started with PrimeRL and implemente…

Unsloth doesn't support distributed training well and doesn't support Kimi models.

I agree full transparency on data adds several other challenges. Still, even releasing the software and infrastructure aspects would be a huge step from where we are now. Also, som…

I'd also add training checkpoints to the list for active transparency. I think the Olmo models do a decent job, but it would be cool to see it for bigger models and for ones that a…

yeah, the costs are definitely a factor and prohibitive in completely replicating an open source model. Still, there's a lot of useful things that can be done cheaply, including fi…