Live data from Hacker News

Model training diary/journal for LLMs?

news.ycombinator.com

1–2 of 2 posts

Model training diary/journal for LLMs?

#1
About half a year ago, some big tech company released an open-source LLM. What makes that model special is that they made available a model training diary/journal recording everything their engineers did to babysit the training process, e.g. "on day 143, the training loss plateaued, so we decreased the learning rate further". I think it was in a shared Google Doc.

Can you remind me of the name of the company/model?