> some evolved language structures in the brain. That's Chomsky's argument. A small set of constraints for organizing language.
This is testable, right? Put a kid from one language tradition in a spot with a different language tradition, and they won't be able to learn it. Eg. kids with native mandarin speaking parents adopted to native Indo-European parents fail at learning English, and will be better at learning Mandarin than their peers with Indo-European heritage.
I don't think anyone is suggesting that the brain has structures that are specific to a particular language.
It’s odd to see people doomwaving two general reasoning engines. It’s especially hard to parse a dark sweeping condemnation based on…people are investing in it? It doesn’t have the right to assign names to things? Idk what the argument is. My most charitable interpretation is “it cant reason abour anything unless we already said it” which is obviously false.
> one of which is an average 14 year old, the other an honors student college freshman The point is that they're not those things. Yes, language models can produce solutions to language tests that a 14 year old could also produce solutions for, but a calculator can do the same thing in the dimension of math - that doesn't make a calculator a 14 year old.
You may not have said it directly but implied, for example if we said A to B, and B to C, the model would have learned the relation and tell you A will go to C, doesn't mean all the sudden it can reason. It's all already in the language and when it has learned enough of numerous forms of A to B, B to C, the relation it's built makes it to give A to C. Yet A to C may very well be some epiphany that we have never thought about. One advantage is the model never get sloppy, it remembers everything, it may overreact/overthink hence hallucination, but it doesn't overlook things or bias like human do (until alignment of course). This is why we're often surprised by the model, but we probably knew it too jut being blind about certain things sometimes so never made the connection.
Wait so then what is his argument? Because you can always postulate that a large enough computer can simulate every human and therefore can learn stuff too — thus you don’t need a human to learn language, nyeh! Obviously, all that stuff ChatGPT says about feelings and emotions came from humans writing it!
As I understand it, the question was whether humans learn language starting from a "blank slate" or whether there's some meaningful structure built in (that's therefore common across all human languages). Chomsky's argument was that children are not exposed to enough example sentences to learn all the rules they learn. It's an argument from the information content of the corpus made available to the language learner,…
That makes no sense. A “vastly larger corpus” would not have the property of “not being exposed to enough example sentences”. If anything, the amount of sentences is far far more than any human would ever encounter — let alone a child.
The heavy investment is what makes this truth uncomfortable - it does not make this truth true (or false). The point is not so much that we already said it, more that the patterns it encodes and surfaces when prompted are patterns in the written corpus, not of the underlying reality (which it has never experienced). Much like a list of all the addresses in the US (or wherever) will tell you very little about the actu…
>not of the underlying reality (which it has never experienced). You've never experienced the "underlying reality" either.
Sure you did, all animal do. Without language, human would live just fine, evidently all animal live this way, deaf people can live, can reason, can triage, may not be sophisticated but they all the underlying reality in their heads, probably gained from try and fail, experiences.
Humans learn from the structure of the world -- not the structure of language. LLMs cheat at generating text because they do so via a model of the statistical structure of text. We're in the world , it is us who stipulate the meaning of words and the structure of text. And we stipulate new meanings to novel parts of the world daily . What else is an 'iPhone' etc. ? There's nothing in `i P h o n e` which is at all lik…
>Humans learn from the structure of the world -- not the structure of language. No we don't. Humans don't experience or perceive reality. We perceive a nice modification of it and that's after excluding all sense data points we simply aren't capable of perceiving at all. Your brain is constantly shifting and fabricating sense data based on internal predictions and that form the basis of what you call reality. You are…
Yeah because all the other things are innate; visual/spatial awareness, touch, smell, vocalization… Humanoids went millions of years literally learning to navigate 3D space and sense “enough heat, food, water” etc Nomadic tribes had built shared resource depots millennia before language. I can see the color gradients of the trees and feel muscles relax without words. Human language beyond some utilitarian labels just…
> 90% of human communication is unspoken. This is as scientific as the idea of humans just using 10% of our brains.
Nonetheless, it's the primary reason people are called to back to office, the in person face time is valued much more than remote chat/email. It's not a coincidence all leaders favor face time, hiring is done via the process of "interview", literally asking the "view". Need more proof?
The Brave summmarizer says: > Wordcels are people who have high verbal intelligence and are good with words, but feel inadequately compensated for their skill. The term "cel" denotes frustration over being denied something they feel they deserve.1 Shape rotators are people with high visuospatial intelligence but low verbal intelligence, who have an intuition for technical problem-solving but are unable to account for…
Wow, okay, thank you for explaining that. They both sound kind of derogative (I'd guessed as much from -cel) but at least I understand them now.
It seems they are intended to. As the article linked above says, it's part of the culture wars.
There's nothing qualitatively less "in the world" about a language model than a human. Yes, a human has more senses, and is doubtless exposed to huge categories of training data that a language model doesn't have access to - but it's false to draw a sharp dichotomy between knowing what an iPhone looks like, and knowing how people talk about iPhones. Consider two people - one, a Papau New Guinea tribesperson from a pr…
> Consider two people - one, a Papau New Guinea tribesperson who is allowed to handle a powered-down iPhone, and told it is an "iPhone", but is otherwise ignorant of its behavior - the other, a cross-platform mobile software developer who has never actually held a physical iPhone, but is intimately familiar with its build systems, API, cultural context etc. Between the two of them, who better understands what an iPho…
No doubt. I don't think anyone's arguing that LLMs have richer, deeper understanding of anything just yet. On the other hand I also don't think it would prove much to vaguely connect a language model to a robot arm and then ask it to do non-language tasks.
> 90% of human communication is unspoken. This is as scientific as the idea of humans just using 10% of our brains.
Nonetheless, it's the primary reason people are called to back to office, the in person face time is valued much more than remote chat/email. It's not a coincidence all leaders favor face time, hiring is done via the process of "interview", literally asking the "view". Need more proof?
Yeah, definitely needs more proof.
Just because it intuitively makes sense doesn’t make it correct or the 90% figure accurate.
doesn't this make LLM a dead end towards AGI and mostly just a neat specific trick?
I think AGI is a questionable concept. We still don't have a good definition of what intelligence really is, and some people keep moving the goal posts. What we need is AI that fills specific needs we have.
If we simply AGI to be "general purpose AI", then my argument is - maybe the approach of LLMs works fine enough for textual generation, but it is not a path towards "general purpose AI".. and what we are going to have is different approaches for different niche use cases.
I'm less convinced there's any unified solution for "general purpose AI" before us here.