Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

371–380 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#371
post #324

Earlier quoted context omitted.

Which those things aren't. In fact they cost considerably more than hiring someone.

LLMs cost significantly less than even a high schooler

Just because for now they are burning money and it's priced considerably under what it's costing them.

Which is why I spoke of "cost" not of "price".

They're in the "disrupt" phase. But that's not forever.

Re: Chomsky on what ChatGPT is good for (2023)

#372

Earlier quoted context omitted.

I think the overarching theme that I glean from LLM critics is some kind of visceral emotional reaction, disgust even, with the idea of them, leading to all these proxy arguments and side quests in order to try and denigrate the idea of them without actually honestly engaging with what they are or why people are interacting with them. so what they don't "understand", by your very specific definition of the word "unde…

You're thinking like an engineer and not a scientist. It's fine, but don't confuse the two.

perhaps I'm thinking like an engineer, but the people I'm referring to are thinking like priests. I don't know who in this debate is playing the role of scientist, but it surely isn't the people parroting irrelevant mock-philosophical quibbles and semantics

Re: Chomsky on what ChatGPT is good for (2023)

#373
post #309

Earlier quoted context omitted.

In Europe he is quite a controversial figure even on the left part of political spectrum- mainly because of his takes on Srebrenica genocide (and recently on the Ukraine war).

News to me. Source?

I think since there was no reply it was just a made up statement.

Re: Chomsky on what ChatGPT is good for (2023)

#374

Earlier quoted context omitted.

I think it does. I think LLM showed us possibility that maybe there's no language but just pile of memes and supplemental compression scheme that is grammar. LLM had really destroyed Chomsky's positions in multiple different ways: nothing perform even close to LLM in language generation, yet it didn't grow a UG for natural languages, while it did develop a shared logic for non-natural languages and abstract concepts,…

>LLM had really destroyed Chomsky's positions in multiple different ways: nothing perform even close to LLM in language generation, yet it didn't grow a UG for natural languages Do you even understand Chomsky's position?

To be honest, I don't, at least not entirely. Noam Chomsky to me is patron saint of compilers and apparent sources of quotes used to justify eye-rolling decisions regarding i18n. At least a lot of his followers' understanding is that the UG is THE UG and a Universal Syntax, and/or is a decisive and scientific refutation of Sapir-Whorf hypothesis as well as European structuralism, not whatever his later works on UG that progressively pivoted its definition or nature vs nurture debates were "meant" to be discussing.

To me this text look like his Baghdad Bob moment. Silly but right and noble. What else is it?

Ironically these days you can just throw this text at ChatGPT to have it debloat or critique text like this transcripts. Worse results than taking time reading yourself, but gives you validation if that is what is needed.

Re: Chomsky on what ChatGPT is good for (2023)

#375

I'm noticing that leftists overwhelmingly toe the same line on AI skepticism, which suggests to me an ideological motivation.

This is a great way to remove any nuance and chance of learning from a conversation. Please don't succumb to black-and-white (or red-and-blue) thinking, it's harmful to your brain.

You're projecting.

Re: Chomsky on what ChatGPT is good for (2023)

#376
post #368
post #99

The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…

Why would that thrill linguists? I'm not saying it hasn't/wouldn't/shouldn't, but I don't see why this technology would have the dramatic impact you imagine. Is/was the same true for ASCII/Smalltalk/binary? They are all another way to translate language into something the computer "understands" . Perhaps the fact that it hasn't would lead some to question the validity of their claims. When a scientist makes a claim a…

> Is/was the same true for ASCII/Smalltalk/binary? They are all another way to translate language into something the computer "understands".

That's converting characters into a digital representation. "A" is represented as 01000001. The tokenization process for an LLM is similar, but it's only the first step.

An LLM isn't just mapping a word to a number, you're taking the entire sentence, considering the position of the words and converting it into vectors within a 1,000+ dimensional space. Machine learning has encoded some "meaning" within these dimensions that goes far far beyond something like an ASCII string.

And the proof here is that the method actual works, that's why we have LLMs.

Re: Chomsky on what ChatGPT is good for (2023)

#377
post #99

The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…

Unfortunately you've undermined your point by making sweeping claims about something that is the literal hardest known problem in philosophy (consciousness). I'm not actually comfortable saying that LLMs aren't conscious. I think there's a decent chance they could be in a very alien way. I realize that this is a very weird and potentially scary claim for people to parse but you must understand how weird and scary con…

Note that I didn't say they aren't conscious, I said they aren't conscious "in the same way as a human being". I left open the possibility they could be conscious "in a very alien way".

Re: Chomsky on what ChatGPT is good for (2023)

#378
post #167

Earlier quoted context omitted.

Dogs out of all creatures probably actually do have some parsing going on for human language. They learn it like we do, picking up context from the environment, actions, tone, facial expressions, body language, etc. You can say 'what's that' in many different ways and a clever dog will react differently for each, even if it's the first time it's heard you say 'what's that?' In a scared tone it'll still react differen…

Parsing? I don't think dogs "parse" language in a decompositional matter.

If they don't parse words to form an understanding of their meaning what are they doing?

Re: Chomsky on what ChatGPT is good for (2023)

#379

Earlier quoted context omitted.

There is a difference between supporting one research paradigm over another and rejecting science altogether to focus on engineering. The first quote and the context around it implies the latter. The success of naval vessels shows it’s possible to navigate without innate star and wind comprehension, so maybe we should think of that inner stuff as phlogiston. (Yeah, this analogy isn’t as nice but it’s quite hard to tr…

Do you not genuinely not understand the logic of the argument? The fact that A does Y using Z doesn't entail that B does Y using Z.

I genuinely don’t understand how the analogy about naval vessels is a fair simplification of the argument that Chomsky’s research programme is, heuristically, a dead-end and should be abandoned. What is A,B,Y,Z?

It’s not like it’s an outrageous position. Chomsky’s tradition is quite controversial and is outside of mainstream nowadays. And connectionism is a valid scientific approach.

Re: Chomsky on what ChatGPT is good for (2023)

#380
post #346

Earlier quoted context omitted.

There actually is a clean line between flying and not flying. And that's when the lift generated is greater than the pull of earth's gravity. The fact that it "feels" weightless gradually doesn't change the fact that if lift weight, plane is flying. There is no "semi flying". If it's already airborne and lift becomes less than weight, then it stops flying and starts gliding. The lift is an emergent behavior of molecu…

Of course, but the cutoff I one of perception more than physics. The airplane is “not flying” right up until the lift generated is infinitesimally more than the weight of the aircraft. Likewise, during “flight” there are times when the lift is less than the weight, during descent. So the line seems clear but it is a matter of degree. The aircraft is not doing anything fundamentally different during the takeoff roll t…

"straight and level flight implies that lift equal to weight == flying too
Post reply on HN