Github was an early OpenAI design partner. OpenAI developed a custom LLM for them. It's so interesting that even after that early mover advantage they have to go back to the foundation model providers. Does this mean that future tech companies have no choice but to do this?
Yes, because transfer learning works. A specialized model for X will be subsumed by a general model for X/Y/Z as it becomes better at Y/Z. This is why models which learn other languages become better at English.
Custom models still have use cases, e.g. situations requiring cheaper or faster inference. But ultimately The Bitter Lesson holds -- your specialized thing will always be overtaken by throwing more compute at a general thing. We'll be following around foundation models for the foreseeable future, with distilled offshoots bubbling up/dying along the way.