Earlier quoted context omitted.
I don’t think that’s a viable strategy. It is very very hard and not many people can do it. Just look at how much Meta is paying to poach the few people in the world capable of training a next gen frontier model.
Why are there actually only a few people in the world able to do this? The basic concept is out there. Lots of smart people studying hard to catch up to also be poached. No shortage of those I assume. Good trainingsdata still seems the most important to me. (and lots of hardware) Or does the specific training still involves lots of smart decisions all the time? And those small or big decisions make all the difference…
But the truth is to have experience building models at this scale requires working at a high level job at a major FAANG/LLM provider. Building what Meta needs is not something you can do in your basement.
The reality is the set of people who really understand this stuff and have experience working on it at scale is very, very small. And the people in this space are already paid very well.