Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Found an Neuron in GPT-2

clementneo.com

Post reply on HN
Print view View original

171–177 of 177 posts

Re: We Found an Neuron in GPT-2

#171

Post by hackinthebochs »

psychphysic wrote: post #158
HarHarVeryFunny wrote: post #157

Earlier quoted context omitted.

Sure, but the point being discussed is that despite the word by word output, the output does not appear to be "chosen" on a word by word basis. OP investigated the case where the word "an" anticipates the following word ("an apple" vs "a pear").

Its chosen by the ngram and randomly so, that does suggest it is completing the text a word at a time.

It's easy to see that its not just doing one token at a time but is anticipating future tokens. Consider the context of a Q&A. The response might start with any of a number of words, exactly which word depends on what comes after. But if it randomly chooses the wrong word, it will either be forced to complete the wrong answer, or be backed into a corner and engage in circumlocutions to course-correct. This doesn't happen in practice for recent big models.

Re: We Found an Neuron in GPT-2

#172

Post by kilgnad »

solarmist wrote: post #164
kilgnad wrote: post #86

Earlier quoted context omitted.

I'm outdated on this research. I took a linguistics class in UCLA about a decade ago and at the time universal grammar was the prevailing norm. Has it changed since then? Have they found a significant number of human civilizations that use divergent grammars?

Chomsky’s universal grammar is a modification of Harris’s Operator grammar. Harris’s operator grammar was based on set theory however Chomsky was enamored with formal logic and ran in that direction. Also, Chomsky became famous while Harris didn’t. Operator grammar is self-discoverable and he published a full description of English grammar in the 1980’s using this theory and an extension in the 1990’s which generaliz…

Well the model needs to be invalidated by science not by elegance or logic. Which grammar is actually more universal or is it a case where both can be applied. In the class I attended Chmosky's stuff was taught as foundational linguistics.

Re: We Found an Neuron in GPT-2

#173

Post by solarmist »

kilgnad wrote: post #172
solarmist wrote: post #164

Earlier quoted context omitted.

Chomsky’s universal grammar is a modification of Harris’s Operator grammar. Harris’s operator grammar was based on set theory however Chomsky was enamored with formal logic and ran in that direction. Also, Chomsky became famous while Harris didn’t. Operator grammar is self-discoverable and he published a full description of English grammar in the 1980’s using this theory and an extension in the 1990’s which generaliz…

Well the model needs to be invalidated by science not by elegance or logic. Which grammar is actually more universal or is it a case where both can be applied. In the class I attended Chmosky's stuff was taught as foundational linguistics.

I’m not sure what you’re arguing.

Any theory that explains all the facts is valid, there can be more than one competing theory. Fame of the author absolutely matters and so does elegance of the theory.

Chomsky has held on so long because it did pretty well and he became very famous, but over time it has needed larger and larger patches to cover its flaws, so other explanations are gaining traction once again.

Re: We Found an Neuron in GPT-2

#174

Post by kilgnad »

solarmist wrote: post #173
kilgnad wrote: post #172

Earlier quoted context omitted.

Well the model needs to be invalidated by science not by elegance or logic. Which grammar is actually more universal or is it a case where both can be applied. In the class I attended Chmosky's stuff was taught as foundational linguistics.

I’m not sure what you’re arguing. Any theory that explains all the facts is valid, there can be more than one competing theory. Fame of the author absolutely matters and so does elegance of the theory. Chomsky has held on so long because it did pretty well and he became very famous, but over time it has needed larger and larger patches to cover its flaws, so other explanations are gaining traction once again.

Hold on. I'm not biased here. I already admitted my knowledge is outdated. I am actively welcoming new information. I am not arguing.

No the elegance of the theory does not matter. The truth of the theory does. I'm saying does the universal grammar you describe, is it actually universal? Amongst all the languages in the world, do they all share this other grammar you describe? If not then the theory is incorrect. It doesn't matter how elegant the theory is.

I brought this up specifically because your reasoning wasn't "scientific" it was more along the lines of logical elegance.

>Chomsky has held on so long because it did pretty well and he became very famous, but over time it has needed larger and larger patches to cover its flaws, so other explanations are gaining traction once again.

See this makes sense and is basically the answer to the question I am asking. So you're basically saying that there were flaws? As in he had to make up new rules constantly because the science was contradicting his theory. Am I correct in this characterization?

Re: We Found an Neuron in GPT-2

#175

Post by solarmist »

kilgnad wrote: post #174
solarmist wrote: post #173

Earlier quoted context omitted.

I’m not sure what you’re arguing. Any theory that explains all the facts is valid, there can be more than one competing theory. Fame of the author absolutely matters and so does elegance of the theory. Chomsky has held on so long because it did pretty well and he became very famous, but over time it has needed larger and larger patches to cover its flaws, so other explanations are gaining traction once again.

Hold on. I'm not biased here. I already admitted my knowledge is outdated. I am actively welcoming new information. I am not arguing. No the elegance of the theory does not matter. The truth of the theory does. I'm saying does the universal grammar you describe, is it actually universal? Amongst all the languages in the world, do they all share this other grammar you describe? If not then the theory is incorrect. It…

Ah, gotcha. Yeah, I didn’t really get what you were trying to say.

I’m not an expert, but if I recall I think the big flaw was insistence on determinism.

And to my knowledge operator grammar is universal to the degree there isn’t a counter example in the dozen of languages he explored and hundreds he had others help him with. But to my knowledge the only language he fully mapped was English.

Edit: take a look at the article on Wikipedia.

https://en.m.wikipedia.org/wiki/Operator_grammar

Re: We Found an Neuron in GPT-2

#176

Post by dgut »

IIAOPSW wrote: post #113
dgut wrote: post #61

Earlier quoted context omitted.

grammar as we know it was devised for the Latin language and linguists spend most of the time attempting to fit other languages into neat boxes that the Latin grammar wasn't designed for. This of course leads to absurdity. Chomsky attempted to solve this problem with his universal grammar, but that too stops working quickly once you get outside of European languages. That is, ignoring linguistic tools is one of the r…

That is not what universal grammar was about. Literally read even the wikipedia page or watch like any lecture on the topic.

What exactly did I write that is wrong? UG argues a universal grammar exists, and that humans innately posses knowledge about this grammar. It's this grammar that enables humans to learn a language (according to UG). They (UG linguists) have created a system of syntax rules that attempts to describe any language, but failed at doing so once they step outside of Indo-European languages. This is partly because Chomsky was hired by MIT to solve machine translation and putting language into a set of neat boxes was his best idea and partly because Chomsky himself had little knowledge of other languages. It's pseudoscience.

Re: We Found an Neuron in GPT-2

#177

Post by IIAOPSW »

dgut wrote: post #176
IIAOPSW wrote: post #113

Earlier quoted context omitted.

That is not what universal grammar was about. Literally read even the wikipedia page or watch like any lecture on the topic.

What exactly did I write that is wrong? UG argues a universal grammar exists, and that humans innately posses knowledge about this grammar. It's this grammar that enables humans to learn a language (according to UG). They (UG linguists) have created a system of syntax rules that attempts to describe any language, but failed at doing so once they step outside of Indo-European languages. This is partly because Chomsky…

Ok now you've made it mostly accurate credit where its due. This is not what you first said.

>grammar as we know it was devised for the Latin language and linguists spend most of the time attempting to fit other languages into neat boxes that the Latin grammar wasn't designed for....Chomsky tried to . All solve this problem with Universal Grammar.

Well first of all, every language has some form of grammar. If there were no rules that made meaning depend on word order, there would be nothing different between what the cat ate and what ate the cat. To claim Grammar is some Latin/Western system being unduly applied is absurd.

You are correct UG was invented as a to explain something about English. Howevert was not an attempt to solve "this problem ("this problem" being conforming to expected systems of Grmmar.) The problem he was interested in solving was about the ability to learn language rapidly despite not having many negative examples of how not to talk. Pay attention here researchers, the lesson maybe extends to you too soon.

UG also isn't a specific enough thing you can check against in a directory. Its a theory, namely a theory that if you study language features you'll eventually discver some things are invaiant. Using it directly and immediately based on nothing but Chomsky's conjecture...all I can focus on passing out at screen goodnightn

Post reply on HN