The leaked Google memo "We have no moat, and neither does OpenAI" is instructive here: https://www.semianalysis.com/p/google-we-have-no-moat-and-ne... Original author is an ML researcher, and the crux of his argument is that most weights in a LLM are significantly overdetermined. Once you have ingested several terabytes of natural language, you know how to generate natural language. The remaining misses are facts tha…
One thing to consider is that while the large transformers used in LLMs might have these diminishing returns, we don't know what discrete jump in model architecture might come next. That model might gain a lot from even more training data. And might gain more from the semi structured data on reddit than the slightly less structured data on Wikipedia and Twitter Maybe
> we don't know what discrete jump in model architecture might come next
This is true, but as long as these models are based on creating sentences based on what sentences that it's seen have looked like, and not on fetching and understanding verifiable facts, these services will do more harm than good overall.
If anything, these models need two sources of training data:
1. The standard language model (as it is now) to be able to generate and process queries and provide understandable answers
2. A database of verifiable factual information that it can query in order to prevent it from completely hallucinating information and then asserting that it is verifiable factual information when asked [0].
Until we can solve the AI hallucination problem, these systems are going to require users to be much more careful with information they're given than most people can manage right now.
[0] https://www.cbsnews.com/news/lawyer-chatgpt-court-filing-avi...