Earlier quoted context omitted.
These new models could be the downfall of authoritarian regimes. These regimes rely on suppressing access to information but soon we'll have an LLM that can answer any questions about the world and fit on a USB stick that can be passed from person to person. No more great firewall of China.
Encyclopedias exist today as well, and already fit on USB sticks. Still, Britannica has yet to topple the Chinese political system, and ChatGPT will not succeed either. Modern authoritarian states are extremely adept at controlling the spread of information. LLMs are not even a drop in the bucket for what the state apparatus can do. If sharing USB sticks with LLMs that talk about the Tiananmen massacre ever became po…
There are several differences between LLMs and encyclopedias
- LLMs don't need specific text about specific events in Chinese. They can be trained in text in English and produce answers in Chinese via translation.
- LLMs probably need much more text to corrupt their veracity -- yet to be tested properly
- LLMs are more obfuscated in their purpose. If I'm sharing a text about a specific event then that is pretty damning, but if an LLM can answer questions about an event then you can't prove that was why I was sharing it. In fact it would be pretty hard to exclude information about specific events from an LLM