Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

221–230 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#221

Post by Workaccount2 »

xwolfi wrote: post #215
AIorNot wrote: post #187

Earlier quoted context omitted.

LLM models are to a large extent neuronal analogs of human neural architecture - of course they reason The claim of the “stochastic parrot” needs to go away Eg see: https://www.anthropic.com/news/golden-gate-claude I think the rub is that people think you need consciousness to do reasoning, I’m NOT claiming LLMs have consciousness or awareness

They are really not neuronal analogs, reasoning is far from what they do. If they reasoned, they'd stick to their guns more readily, but try to contradict an LLM and it will make any logic leap you ask it too. If you debate with me, I'll keep reasoning on the same premises and usually the difference between two humans is not in reasoning but in choice of premises. For instance you really want here to assert that LLM…

Gemini 2.5 will tell you when you're wrong. It's the first model to do so.

Re: Chomsky on what ChatGPT is good for (2023)

#222

Post by lostmsu »

tpm wrote: post #138
lostmsu wrote: post #104

Earlier quoted context omitted.

I don't know what he's talking about. Humans clearly outperform ants in navigation. Especially if you allow arbitrary markings on the terrain. Sounds like "ineffable nature" mumbo-jumbo.

Arbitrary markings on the terrain? Why not GPS, satellite photo etc? All of those are human inventions and we can navigate much better and in a broader set of environments than ants thanks to them.

Because I was making a stronger argument (strict vs relaxed requirements on humans) to preempt another nature mumbo-jumbo argument that GPS is not the same/cheating.

Re: Chomsky on what ChatGPT is good for (2023)

#224

Post by FilosofumRex »

Calavar wrote: post #214
irrational wrote: post #204

I have a degree in linguistics. We were taught Chomsky’s theories of linguistics, but also taught that they were not true. (I don’t want to say what university it was since this was 25 years ago and for all I know that linguistics department no longer teaches against Chomsky). The end result is I don’t take anything Chomsky says seriously. So, it is difficult for me to engage with Chomsky’s ideas.

I don't have a degree in linguistics, but I took a few classes about 15 years ago, and Chomsky's works were basically treated as gospel. Although my university's linguistics faculty included several of his former graduate students, so maybe there's a bias factor. In any case, it reminds me of an SMBC comic about how math and science advance over time [1] [1] https://smbc-wiki.com/index.php/How-math-works

Linguistics has been largely subsumed by CS (LLM, speech synthesis, translation). It's not an empirical science or social science and most of its theories are not falsifiable.

But generally speaking Chomsky's ideas, and in particular, the Universal Grammar are no longer in vogue.

Re: Chomsky on what ChatGPT is good for (2023)

#225

Post by lenkite »

loveparade wrote: post #201
atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a little bit. While it has not helped us to understand our own intelligence, it has made me question whether such a thing even exists. Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and…

It is possible that we simply haven't yet discovered those natural laws for "emergent behavior" from the "messy substrate".

Re: Chomsky on what ChatGPT is good for (2023)

#226

Post by lostmsu »

gf000 wrote: post #109
lostmsu wrote: post #102

Earlier quoted context omitted.

I think GPT-2 (2019) was already strong enough argument for possibility of modeling knowledge and language that Chomsky rejected.

Though given that LLMs fundamentally can't know whether they know something or not (without a later pass of fine-tuning on what they should know) is a pretty good argument against them being good knowledge bases.

No, it is not. In mathematical limit this applies to literally everything. In practice you are not going to store video compressed with a lossless codec, for example.

Re: Chomsky on what ChatGPT is good for (2023)

#227

Post by dropofwill »

windexh8er wrote: post #207
irrational wrote: post #204

I have a degree in linguistics. We were taught Chomsky’s theories of linguistics, but also taught that they were not true. (I don’t want to say what university it was since this was 25 years ago and for all I know that linguistics department no longer teaches against Chomsky). The end result is I don’t take anything Chomsky says seriously. So, it is difficult for me to engage with Chomsky’s ideas.

I'm rather confused by this statement. I've read a number of Chomsky pieces and have listened to him speak a number of times. To say his theories were all "not true" seems, to an extent, almost impossible. Care to expand on how his theories can be taught in such a binary way?

Generally what people are talking about are his universal grammar or generative syntax theories/approaches, which are foundational to how you approach many topics. Because you build your academic career based on specialization they are hotly contested (for the material reasons of jobs, funding, tenure, etc.).

This leads to people who agree hiring each other and departments ‘circling the wagon’ on these issues. You’ll see this referred to as east vs west coast, but it’s not actually that clearly geographically delineated.

So anyways, these are open questions that people do seriously discuss and study, but the politics of academia make it difficult and unfortunately this often trickles down to students.

Re: Chomsky on what ChatGPT is good for (2023)

#228

Post by dmvdoug »

LudwigNagasena wrote: post #82

That was a weird ride. He was asked whether AI will outsmart humans and went on a rant about philosophy of science seemingly trying to defend the importance of his research and culminated with some culture war commentary about postmodernism.

There are lots of stories about Chomsky ranting and wielding his own disciplinary authority to maintain himself as center of the field.

Re: Chomsky on what ChatGPT is good for (2023)

#229

Post by majormajor »

chillingeffect wrote: post #208
CamperBob2 wrote: post #173

Earlier quoted context omitted.

They do. Spend some time using a modern reasoning model. There is a class of interesting problems, nestled between trivial ones whose answers can simply be regurgitated and difficult ones that either yield nonsense or involve tool use, that transformer networks can absolutely, incontrovertibly reason about.

I have been able to get chatgpt to synthesize in the edges of two domains in ideaspace, say, psychology and economics, but surprisingly it struggled helping me write ODE code in go. In the first case, I think it actually synthesized. In the latter it couldn't extrapolate enough ideas from the two fields into one.

How can you distinguish "I think it did something really impressive in the first case but not the second" from "it spat out something that looked interesting in both cases but in the latter case there was an objective criteria that exposed a lack of true understanding"?

It's famously easier to impress people with soft-sciences speculation than it is to impress the rules of math or compilers.

Re: Chomsky on what ChatGPT is good for (2023)

#230

Post by Calavar »

FilosofumRex wrote: post #224
Calavar wrote: post #214

Earlier quoted context omitted.

I don't have a degree in linguistics, but I took a few classes about 15 years ago, and Chomsky's works were basically treated as gospel. Although my university's linguistics faculty included several of his former graduate students, so maybe there's a bias factor. In any case, it reminds me of an SMBC comic about how math and science advance over time [1] [1] https://smbc-wiki.com/index.php/How-math-works

Linguistics has been largely subsumed by CS (LLM, speech synthesis, translation). It's not an empirical science or social science and most of its theories are not falsifiable. But generally speaking Chomsky's ideas, and in particular, the Universal Grammar are no longer in vogue.

Linguistics is a heterogenous field, an while some parts aren't empirical, others absolutely are. I can attest to that from the journal articles I read in undergrad. They had experimental data and statistical analyses.
Post reply on HN