Earlier quoted context omitted.
> Funny how no-one talks about AI neutrality like we used to discuss net neutrality From my perspective, these LLM providers aren’t infrastructure providers but more like SaaS. And there are also open models that you can use to do anything you want. These AI companies are also under a lot of scrutiny and sometimes it feels like whatever they do in this regard, they’re bound to piss someone off. Last but not least, it…
The irony is the current administration’s posturing against Chinese AI companies forcing something like this is going to actually bolster competitive advantage overseas. That and European companies as well. The landscape is going to change drastically in 5 years once all the data centers are built all over the world. The science behind these models are being worked on IN PUBLIC. The research is not secret. The implem…
Only to a limited extent - the US companies stopped sharing research a long time ago, other than Anthropic's interpretability research (which also seems to have dried up?). Interestingly most of the sharing is now coming from the Chinese side, largely DeepSeek. Ziphu/Z.ai (GLM) is also partner in the Slime RL training framework.
I wouldn't call much, if any, of this "science" - it's all empiricalism. Throw spaghetti at the wall and see what sticks. There's a famous quote from Noam Shazeer:
"We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence"
https://arxiv.org/abs/2002.05202v1
Jakob Uszkoreit has also talked about the empiricalism that it took to make what would become the Transformer, and any complex neural network architecture work.