Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Found an Neuron in GPT-2

clementneo.com

Post reply on HN
Print view View original

41–50 of 177 posts

Re: We Found an Neuron in GPT-2

#41

Post by caf »

vore wrote: post #34
phkahler wrote: post #31

N00b to this. How are the neurons outputs read to produce text? They talk about tokens as if a token is a word. But if token==word then every word would have a specific output and there's nothing to see here. So again, how are neuron outputs converted to letters/text?

It goes something like this: input text -> input tokens -> input embeddings -> model -> output embeddings -> output tokens -> output text Tokens aren't necessarily words: they can be fragments of words and you can check out this behavior here: https://platform.openai.com/tokenizer For instance, "an eagle" is tokenized to [an][ eagle], but "anoxic" is tokenized to [an][oxic], so just looking for the [an] token is not…

Further to this, as I understand it the "embedding" is mapping the tokens into vectors in a space where tokens semantically similar will be close together in the space.

Re: We Found an Neuron in GPT-2

#42

Post by Tao3300 »

caf wrote: post #39
edgyquant wrote: post #19

Earlier quoted context omitted.

I is a vowel so your rule about sound doesn’t apply here. It’s also not a rule I’ve ever heard, in school we’re taught that only vowels get a before them

The word is "historic" either way, the commenter was using " 'istoric " as a way of indicating a particular pronunciation. There's a few words in English that begin with the letter h but not with the consonant sound /h/ (depending on your accent!), mostly because they're French in origin: historic is one, and there's also hour, heir, honour. "It is an honour to be here"

Yeah, where I'm from, since historic has the emphasis on STOR and not HI (as in history), the h is much softer, so it's pretty normal to say "an historic event" but also "a history of the world".

It's silly to act like there's a right or wrong. Dialects and accents are a special thing. Just be consistent.

Re: We Found an Neuron in GPT-2

#43

Post by PeterisP »

caf wrote: post #40
varispeed wrote: post #38

Earlier quoted context omitted.

My friend who is not a native English speaker told me that one thing he struggled the most while learning English was the "a" and "an". He couldn't grasp the concept how it is possible that person knows which determiner to use before saying the word, until he learned it as just the part of the word and then he uses it depending on the context which one can "feel". So when he sees an apple, he says this is an apple. E…

And of course it is not unheard of for a native English speaker to correct themselves if they change their mind about which noun to use. You can imagine being asked to very rapidly verbally classify fruits ("it's an apple, it's an apple, it's a pear, it's an apple,..") that you might well find yourself stumbling like "it's a.. an apple".

This is also why a bunch of words have changed - e.g. apron was originally called napron, but when people were saying "a napron" it got mangled to "an apron"..

Re: We Found an Neuron in GPT-2

#44

Post by KhoomeiK »

LudwigNagasena wrote: post #13

It’s notable how successful LLMs despite the lack of any linguistic tools in their architectures. It would be interesting to know how different a model would be if it operated on eg dependency trees instead of the linear list of tokens. Surely, the question of “a/an” would be solved with ease as the model would be required to come up with a noun token before choosing its determiner. I wonder if the developers of LLMs…

Several papers have explored emergent linguistic structure in LMs. Here's some early introductory work in this space. Despite having explicit syntax parses etc as input, models seem to learn something like syntax.

https://arxiv.org/abs/1905.05950

https://aclanthology.org/N19-1419/

https://arxiv.org/abs/1906.04341

Re: We Found an Neuron in GPT-2

#45

Post by caf »

PeterisP wrote: post #43
caf wrote: post #40

Earlier quoted context omitted.

And of course it is not unheard of for a native English speaker to correct themselves if they change their mind about which noun to use. You can imagine being asked to very rapidly verbally classify fruits ("it's an apple, it's an apple, it's a pear, it's an apple,..") that you might well find yourself stumbling like "it's a.. an apple".

This is also why a bunch of words have changed - e.g. apron was originally called napron, but when people were saying "a napron" it got mangled to "an apron"..

Maybe one day "knife" will become "ife".

Re: We Found an Neuron in GPT-2

#46

Post by hgsgm »

PeterisP wrote: post #43
caf wrote: post #40

Earlier quoted context omitted.

And of course it is not unheard of for a native English speaker to correct themselves if they change their mind about which noun to use. You can imagine being asked to very rapidly verbally classify fruits ("it's an apple, it's an apple, it's a pear, it's an apple,..") that you might well find yourself stumbling like "it's a.. an apple".

This is also why a bunch of words have changed - e.g. apron was originally called napron, but when people were saying "a napron" it got mangled to "an apron"..

And narrow from "an arrow"

Re: We Found an Neuron in GPT-2

#47

Post by hgsgm »

varispeed wrote: post #38
tqi wrote: post #10

> We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vowel or not, but GPT-2 is only capable of predicting one word at a time. We still don’t have a full answer... I'm not sure I understand why this is an open question. While I get that GPT-2 is predicting only one word at a time, it doesn't seem that surprising t…

My friend who is not a native English speaker told me that one thing he struggled the most while learning English was the "a" and "an". He couldn't grasp the concept how it is possible that person knows which determiner to use before saying the word, until he learned it as just the part of the word and then he uses it depending on the context which one can "feel". So when he sees an apple, he says this is an apple. E…

Does his language have grammatical gender?

Re: We Found an Neuron in GPT-2

#48

Post by hgsgm »

vore wrote: post #34
phkahler wrote: post #31

N00b to this. How are the neurons outputs read to produce text? They talk about tokens as if a token is a word. But if token==word then every word would have a specific output and there's nothing to see here. So again, how are neuron outputs converted to letters/text?

It goes something like this: input text -> input tokens -> input embeddings -> model -> output embeddings -> output tokens -> output text Tokens aren't necessarily words: they can be fragments of words and you can check out this behavior here: https://platform.openai.com/tokenizer For instance, "an eagle" is tokenized to [an][ eagle], but "anoxic" is tokenized to [an][oxic], so just looking for the [an] token is not…

Surely "an eagle" is [ an][ eagle] since the an starts a new word.

Re: We Found an Neuron in GPT-2

#49

Post by eightails »

fragmede wrote: post #14
unshavedyak wrote: post #6

I wonder if this stuff will ever be applicable to a person and a laptop (or if it is now?). Ie this seems like such a cool area to be in but the data volumes required are huge, complex, etc. Code is simple, cheap, lean, etc by comparison. Do we have any insight on how this area of research could be usable with less hardware and data? Is there a visible future where a guy and a laptop can make a big program? (without…

Does this hypothetical laptop have a GPU? StableDiffusion is in this realm of "stuff" and is runnable on consumer GPU systems. It's a bit of trouble to get setup if you're not a python dev (and kinda still is if you are) but it's a pretty neat ML model to play around with.

For sure, and even training is very doable on consumer hardware these days. Techniques like Dreambooth and LoRA have dramatically lowered the compute cost of finetuning large models on specific concepts. A recent GPU can train Stable Diffusion models on a concept using LoRA in < 30 minutes.

Re: We Found an Neuron in GPT-2

#50

Post by sunk1st »

varispeed wrote: post #38
tqi wrote: post #10

> We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vowel or not, but GPT-2 is only capable of predicting one word at a time. We still don’t have a full answer... I'm not sure I understand why this is an open question. While I get that GPT-2 is predicting only one word at a time, it doesn't seem that surprising t…

My friend who is not a native English speaker told me that one thing he struggled the most while learning English was the "a" and "an". He couldn't grasp the concept how it is possible that person knows which determiner to use before saying the word, until he learned it as just the part of the word and then he uses it depending on the context which one can "feel". So when he sees an apple, he says this is an apple. E…

The rule is the rule because that’s what make sense phonetically. It’s why you would say “el agua” and “el hacha” in Spanish even though those articles don’t match the gender of those words.
Post reply on HN