Anyway, I think open weight models will win. I am strong believer that AI (even the frontier) can be done just as "deductive core, encyclopedic shell". That is:
- Deductive core is a small number of basic primitives (in an LLM probably less than couple thousands, like words in simple english) which encode the human common sense reasoning logic and epistemology. These might correspond to the elements of the embedding vector as well.
- Encyclopedic shell is a collection of many facts about the world (most importantly their definitions) expressed in the language of deductive core. The new facts can be deduced by applying the rules from deductive core to the encyclopedic shell; that's what we call "model reasoning".
Deductive core, encyclopedic shell is not a new concept. Any programming language has primitives and then a library of useful functions. In Lisp, the deductive core is special forms; in Haskell, it is Haskell Core (based on HM typesystem); in Lean, the core is based on Calculus of Constructions; in Prolog, the core is mostly the resolution algorithm. Even in predicate logic you have axioms and then you build a library of useful tautologies from that.
I also believe that current AI (large reasoning models) deductive cores suck quite badly. They have been generated by an evolutionary algorithm as an emergent property of LLM pretraining, and are probably order or two magnitude larger than actually required. I believe hand-crafted deductive cores will win in the long term.
Now, through this lens, what is there to be export-controlled (or just controlled in general)?
You can't control the encyclopedic shell. It's like Wikipedia or other books that have already been published. It would mean closing down research and STEM publishing.
I also don't think you can control the deductive core. It's just an algorithm (a language interpreter), and desirably, a small one. Any country with a rich enough history of philosophy can probably recreate one (such as China). Moreover, I believe many open source models already contain decent deductive cores so that fight has been lost as well.
The only reason we have this discussion is that LLMs are shrouded in mystery, because their vast knowledge (the encyclopedic shell) is written in an unaccessible language (the deductive core) of their weights.
Every couple years, capitalism tries to reinvent itself by creating a new type of property. Yesterday it was "big data", today it's "model weights". Don't fall for it; it's an enclosure of publicly available knowledge and our ability to use it.