N00b to this. How are the neurons outputs read to produce text? They talk about tokens as if a token is a word. But if token==word then every word would have a specific output and there's nothing to see here. So again, how are neuron outputs converted to letters/text?
We Found an Neuron in GPT-2
31–40 of 177 posts
Re: We Found an Neuron in GPT-2
#32It’s notable how successful LLMs despite the lack of any linguistic tools in their architectures. It would be interesting to know how different a model would be if it operated on eg dependency trees instead of the linear list of tokens. Surely, the question of “a/an” would be solved with ease as the model would be required to come up with a noun token before choosing its determiner. I wonder if the developers of LLMs…
I think the lack of explicit linguistic tools is the key to success, forcing/enabling the generic model to learn implicit linguistic tools (there's some research identifying that analysis of specific linguistic phenomena happens at specific places in the NN layers) that work better than what we could implement. "It would be interesting to know how different a model would be if it operated on eg dependency trees inste…
[deleted]
Re: We Found an Neuron in GPT-2
#33> The choice depends on whether the word that comes after starts with a vowel or not Or an aitch / h, but only sometimes.
More accurately, it's whether the word that comes after starts with a vowel sound . This is why `an 'istoric` is correct, and `a historic` is correct, but `an historic` is incorrect (as famously used by Steven Colbert).
Or "Esau was an hairy man" from beyond the fringe https://youtu.be/UOsYN---eGk
Re: We Found an Neuron in GPT-2
#34N00b to this. How are the neurons outputs read to produce text? They talk about tokens as if a token is a word. But if token==word then every word would have a specific output and there's nothing to see here. So again, how are neuron outputs converted to letters/text?
It goes something like this:
input text -> input tokens -> input embeddings -> model -> output embeddings -> output tokens -> output text
For instance, "an eagle" is tokenized to [an][ eagle], but "anoxic" is tokenized to [an][oxic], so just looking for the [an] token is not sufficient. Therefore, you would need to map the output text all the way back into the model to figure out what neuron(s) in the model would generate "an" over "a". Since the bulk of GPT is all unsupervised learning, any connections it makes in its neural network is all emergent.
Re: We Found an Neuron in GPT-2
#35> We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vowel or not, but GPT-2 is only capable of predicting one word at a time. We still don’t have a full answer... I'm not sure I understand why this is an open question. While I get that GPT-2 is predicting only one word at a time, it doesn't seem that surprising t…
Yeah, my feeling here was that it's sort of tautological: if GPT predicts "a", then it must then predict a word that would follow "a" and not require "an" (and vice versa). And if you think about it from the opposite direction: if it's working out a response that is eventually going to have "apple" in it, then all the data it's trained on is going to cause it to predict "an" even before it needs to predict "apple". (…
I think that's essentially right - if it's already "thinking about" apples, then ' an' will be significantly upregulated at the expense of ' a', then if it does choose to output an ' an' token, ' apple' becomes a highly likely followup.
You could probably test this by seeing if prompts containing a lot of nouns that start with a vowel sound results in output that contains a higher proportion of otherwise unrelated first-vowel nouns. (ie your prompt includes lots of apples, apricots, avocados, asparagus, aubergines, elderberries, eggplants, endives, oranges, olives, okras, onions and you count the proportion of non-food nouns in the result that start with a vowel and non-vowel sound).
Re: We Found an Neuron in GPT-2
#36Earlier quoted context omitted.
I'm always thrown when people write "an SQL parser". I'll bet roughly half this crowd is the opposite and cringe when they read "a SQL parser".
I'm in the special superstate where both make me cringe and wish there was a better way! Solution get rid of article. Someone want to know if thing is definitive thing, can figure it out from context!
There's a third article besides "a" and "an". So in order to solve the problem, we need only to ensure that there is only one SQL parser, so that we can refer to it as "the SQL parser".
(also, I'm in the opposite superstate to the one you are in -- both "a SQL parser" and "an SQL parser" look fine to me, although I internally pronounce the first as "a seekwul parser" and the second as "an ess cue ell parser")
Re: We Found an Neuron in GPT-2
#37Earlier quoted context omitted.
More accurately, it's whether the word that comes after starts with a vowel sound . This is why `an 'istoric` is correct, and `a historic` is correct, but `an historic` is incorrect (as famously used by Steven Colbert).
I'm always thrown when people write "an SQL parser". I'll bet roughly half this crowd is the opposite and cringe when they read "a SQL parser".
an ess kyu el parser is fine
a sequel parser is also fine *
* my old Unix colleagues will remember that Sequel is a brand of database that nobody else remembers anymore so tended to use the initialism.
Re: We Found an Neuron in GPT-2
#38> We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vowel or not, but GPT-2 is only capable of predicting one word at a time. We still don’t have a full answer... I'm not sure I understand why this is an open question. While I get that GPT-2 is predicting only one word at a time, it doesn't seem that surprising t…
My friend who is not a native English speaker told me that one thing he struggled the most while learning English was the "a" and "an". He couldn't grasp the concept how it is possible that person knows which determiner to use before saying the word, until he learned it as just the part of the word and then he uses it depending on the context which one can "feel". So when he sees an apple, he says this is an apple.
Exactly what you mean by bigram.
Re: We Found an Neuron in GPT-2
#39Earlier quoted context omitted.
More accurately, it's whether the word that comes after starts with a vowel sound . This is why `an 'istoric` is correct, and `a historic` is correct, but `an historic` is incorrect (as famously used by Steven Colbert).
I is a vowel so your rule about sound doesn’t apply here. It’s also not a rule I’ve ever heard, in school we’re taught that only vowels get a before them
The word is "historic" either way, the commenter was using " 'istoric " as a way of indicating a particular pronunciation.
There's a few words in English that begin with the letter h but not with the consonant sound /h/ (depending on your accent!), mostly because they're French in origin: historic is one, and there's also hour, heir, honour. "It is an honour to be here"
Re: We Found an Neuron in GPT-2
#40> We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vowel or not, but GPT-2 is only capable of predicting one word at a time. We still don’t have a full answer... I'm not sure I understand why this is an open question. While I get that GPT-2 is predicting only one word at a time, it doesn't seem that surprising t…
My friend who is not a native English speaker told me that one thing he struggled the most while learning English was the "a" and "an". He couldn't grasp the concept how it is possible that person knows which determiner to use before saying the word, until he learned it as just the part of the word and then he uses it depending on the context which one can "feel". So when he sees an apple, he says this is an apple. E…
And of course it is not unheard of for a native English speaker to correct themselves if they change their mind about which noun to use. You can imagine being asked to very rapidly verbally classify fruits ("it's an apple, it's an apple, it's a pear, it's an apple,..") that you might well find yourself stumbling like "it's a.. an apple".