31–40 of 107 posts
LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…
Earlier quoted context omitted.
It’s not a mapping of words. It’s a mapping of character sequences. You can ask chatgpt to define “hackernewsitis (zero hits on google) and it gives a plausible definition. Also the fact that it can write code is evidence that it can understand new concepts. A variable declaration is a coining of a (very short lived) new word.
> It’s not a mapping of words. It’s a mapping of character sequences. This is plainly wrong if you read the word2vec paper.
That conclusion is not correct. Generative models can combine existing concepts in novel ways which have never been considered before. This is most easily seen with text-to-image models. For example the teddy bear swimming in an Olympic pool [1] or many other surreal examples [2] were not part of the training set. So as long as an answer can be expressed as a combination of primitive concepts, LLMs can generate them.
[1] https://imagen.research.google/main_gallery_images/teddy-bea...
The article is claiming that advanced AIs cannot become more intelligent than humans, essentially because LLMs cannot become more intelligent than humans. LLMs aren’t the be all and end all of AI though. They’re an impressive but inherently limited stepping stone, with a very constrained scope of applicability and capabilities. There is no reason to suppose that future, much more advanced architectures can’t surpass…
I think that's false - you could imagine it playing at GM level without making mistakes that real GMs make
But it's missing an important fact: chess ais aren't limited to learning only from human games. They can learn from self play.
You can extend this to physics (if we can simulate physics precisely enough, there's no reason ai can't make fundamental discoveries in physics)
This is an old discussion with much deeper insights already available than I can parrot here, but one obvious counter just to this argument is that even an LLM can be an existential threat, depending what it can convince you it knows.
LLMs don't learn to simulate or mimic, that's just a byproduct. They learn to predict the training corpus. There is absolutely nothing about the act of prediction that necessitates an upper bound of intelligence on the corpus itself. https://www.pnas.org/doi/full/10.1073/pnas.2016239118 They found representations on fundamental properties of proteins such as secondary structure, contacts, and biological activity in a…
I myself have started doing this. For example, I don’t publish code on GitHub anymore to prevent copilot training on my own code. Normally I like to get paid for work, instead of paying for GitHub and doing work for them:) I’d much rather take a page out of an artists playbook and require royalties.
I have also built a coding test harness to test for AGI and let me tell you, that the LLMs are completely unable to reason on the most intermediate and advanced algorithmic problems you present to them the moment they are not in the corpus. This involves a nearly complete lack of the ability to generalize and infer new concepts needed to solve the problem even if the concept is explained in the problem description.
Now the issue is that it is impossible to present the test because once it ends up in the corpus, then someone will put the solution in the corpus as well and claim the ability to reason. You can believe me or not; I won’t be making my test available either way, but it is not too hard to make one of your own to convince yourself.
I wish it was possible to make the tests like that available but alas we will likely end up with a medical-style double blind kind of tests to get rid of AI FUD.
