121–130 of 389 posts
It’s time to stop writing in this elitist jargon. If you’re communicating and few people understands you, then you’re a bad communicator. I read the whole thing and thought: wait, was there a new thought or interesting observation here? What did we actually learn?
Earlier quoted context omitted.
> LLMs do surface real linguistic structure... It's completely irrelevant because the point he's making is that LLMs operate differently from human languages as evidenced by the fact that they can learn language structures that humans cannot learn. Put another way, I'm sure you can point out an infinitude of similarities between human language faculty and LLMs but it's the critical differences that make LLMs not usef…
> You can pre-train an LLM on English or an impossible language and they do equally well It's impressive that LLMs can learn languages that humans cannot. In what frame is this a negative? Separately, "impossible language" is a pretty clear misnomer. If an LLM can learn it, it's possible.
That's what "impossible language" means in this context, not something like computationally impossible or random.
Chomsky’s notion is: LLMs can only imitate, not understand language. But what exactly is understanding? What if our „understanding“ is just unlocking another level in a model? Unlocking a new form of generation?
If we define "understanding" like "useful", as in, not an innate attribute, but something in relation to a goal, then again, a good imitation, or a rudimentary model can get very far. ChatGPT "understood" a lot of things I have thrown at it, be that algorithms, nutrition, basic calculations, transformation between text formats, where I'm stuck in my personal development journey, or how to politely address people in the email I'm about to write.
>What if our „understanding“ is just unlocking another level in a model?
I believe that it is - that understanding is basically an illusion. Impressions are made up from perceptions and thinking, and extrapolated over the unknown. And just look how far that got us!
Earlier quoted context omitted.
> You can pre-train an LLM on English or an impossible language and they do equally well It's impressive that LLMs can learn languages that humans cannot. In what frame is this a negative? Separately, "impossible language" is a pretty clear misnomer. If an LLM can learn it, it's possible.
The latter. Moro showed that you can construct simple language rules, in particular linear rules, like the third word of every sentence modifies the noun, that humans have a hard time learning (specifically they use different parts of their brain in MRI scans and take longer to process than control languages) and are different from conventional human language structure (which hierarchical structure dependent, i.e. ro…
> The world’s preeminent linguist Noam Chomsky, and one of the most esteemed public intellectuals of all time, whose intellectual stature has been compared to that of Galileo, Newton, and Descartes, tackles these nagging questions in the interview that follows. By whom?
In all seriousness tho, not much of anything he says is taken seriously in an academic sense any more. Univeral Grammar, Minimalism, etc. He's a very petty dude. The reason he doesn't engage with GPT is because it suggests that linguistic learning is unlike a theory he spent his whole life [unsuccessfully] promoting, but he's such a haughty know-it-all, that I guess dummies take that for intelligence? It strikes me a…
As this is Hacker News, it is worth mentioning that he developed the concept of context-free grammars. That is something many of us encounter on a regular basis.
No matter what personality flaws he might have and how misguided some of his political ideas might be, he is one of the big thinkers of the 20th century. Very much unlike Trump.
Earlier quoted context omitted.
The latter. Moro showed that you can construct simple language rules, in particular linear rules, like the third word of every sentence modifies the noun, that humans have a hard time learning (specifically they use different parts of their brain in MRI scans and take longer to process than control languages) and are different from conventional human language structure (which hierarchical structure dependent, i.e. ro…
Ok then .. what makes that a negative? You're describing a human limitation and a strength of LLMs
As I said in another comment this whole dispute would be put to bed if people understood that they don't care about what humans do (and that Chomsky does).
Earlier quoted context omitted.
> Running tens of thousands of chips crunching a bajillion numbers a second sounds fun, but that's not automatically "engineering". Not if some properties are unexpectedly emergent. Then it is science. For instance, why should a generic statistical model be able to learn how to fill in blanks in text using a finite number of samples? And why should a generic blank-filler be able to produce a coherent chat bot that ca…
> because it would need impossible amounts of data The training data for LLM is so massive that it reaches the level of impossible if we consider that no person can live long enough to consume it all. Or even a small percent of it. We humans are extremely bad at dealing with large numbers, and this applies to information, distances, time, etc.
Earlier quoted context omitted.
"Tom Jones" isn't a strawman, Chomsky is addressing an actual argument in a published paper from Steven Piantadosi. He's using a pseudonym to be polite and not call him out by name. > instead of even attempting to summarize the arguments for his position.. He makes a very clear, simple argument, accessible to any layperson who can read. If you are studying insects what you are interested in is how insects do it not w…
Is it polite to deprive readers of context necessary to understand what the speaker is talking about? I was also very confused by that part and I had no idea whom or what he was talking about or why he even started taking about that. I searched for an actual paper by that guy because you’ve mentioned his real name. I found “Modern language models refute Chomsky’s approach to language”. After reading it seems even mor…
Lol. It's clear you are not interested in having any kind of rational discussion on the topic and are driven by some kind of zealotry when you claim to have read a technical 40 page paper (with an additional 18 pages of citations) in 30 minutes.
Even if by some miraculous feat you had read it you haven't made a single actual argument or addressed any of the points made by Chomsky.
Earlier quoted context omitted.
Ok then .. what makes that a negative? You're describing a human limitation and a strength of LLMs
It's not a negative, it's just not what humans do, which is Chomsky's (a person studying what humans do) point. As I said in another comment this whole dispute would be put to bed if people understood that they don't care about what humans do (and that Chomsky does).
It's completely unremarkable that humans are unable to learn certain languages, and soon it will be unremarkable when humans have no cognitive edge over machines.
Response: Science? "Ancient Linguistics" would more accurately describe Chomsky's field of study and its utility
Earlier quoted context omitted.
It's not a negative, it's just not what humans do, which is Chomsky's (a person studying what humans do) point. As I said in another comment this whole dispute would be put to bed if people understood that they don't care about what humans do (and that Chomsky does).
Suggestion for you then, in your first response you would have been clearer to say "The reason Chomsky seems like such a retard here, is because he clings to irrelevant nonsense" It's completely unremarkable that humans are unable to learn certain languages, and soon it will be unremarkable when humans have no cognitive edge over machines. Response: Science? "Ancient Linguistics" would more accurately describe Chomsk…
If science is irrelevant to you it's you who should have recognized this before spouting off.