Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We Found an Neuron in GPT-2

clementneo.com

Post reply on HN
Print view View original

101–110 of 177 posts

Re: We Found an Neuron in GPT-2

#101

Post by jetsnoc »

Thorentis wrote: post #94
SamBam wrote: post #80

Interestingly I feel like humans have this as well, sometimes. Sometimes if someone is working though a complex thought and they're not really sure where they're going, they'll pause while thinking of the word they want to use, and might sound like "the discussion is an... an... epistemological one" Obviously they may have been conscious that the next word was going to start with "epi..." and they are just trying to…

When you say "they have no idea about what they're about to say" you're talking about conscious thought. I think there is a difference between rational thought (thinking by going through a series of logically connected steps) and intuition, where you can arrive at a conclusion or knowledge of some fact or concept or knowing how to do something, without having gone through those conscious steps. Does one count as "tho…

Intuition is another cognitive process honed by training and reinforcement of neurons through all sorts of sensories and feedback to our brain. I think parallel models not unlike additional cognitive processes in our brain assisting the NLG will eventually make it more similar to how our cognitive processes actually work.

Disclaimer: Uneducated opinion on my behalf, I'm a hobbyist only.

Re: We Found an Neuron in GPT-2

#102

Post by nl »

fragmede wrote: post #14
unshavedyak wrote: post #6

I wonder if this stuff will ever be applicable to a person and a laptop (or if it is now?). Ie this seems like such a cool area to be in but the data volumes required are huge, complex, etc. Code is simple, cheap, lean, etc by comparison. Do we have any insight on how this area of research could be usable with less hardware and data? Is there a visible future where a guy and a laptop can make a big program? (without…

Does this hypothetical laptop have a GPU? StableDiffusion is in this realm of "stuff" and is runnable on consumer GPU systems. It's a bit of trouble to get setup if you're not a python dev (and kinda still is if you are) but it's a pretty neat ML model to play around with.

Stable Diffusion runs really well on M1 GPU.

Re: We Found an Neuron in GPT-2

#103

Post by furyofantares »

justusthane wrote: post #100
furyofantares wrote: post #98

Earlier quoted context omitted.

I never know what I'm about to say, but somehow coherent sentences come out. I certainly don't know what words a sentence is going to end with when I'm thinking or saying the first words in the sentence. I just think or say the sentence from start to finish, never knowing what the next word is going to be as I'm thinking the current one, and by the end of it I've thought or said a full sentence that makes sense.

This is so weird. I am very “In my head” and have generally planned what I’m going to say before I open my mouth. It makes me feel like I’m not very good at conversation though, especially smalltalk.

Yeah, sometimes I think it in advance, in which case as I'm thinking it, I don't usually know the next word and still arrive at a coherent sentence by the end, just plowing forward from start to finish.

Re: We Found an Neuron in GPT-2

#104

Post by johntb86 »

Is the " an" token the only way GPT-2 will ever produce the string " an"? Or will it sometimes combine separate " a" "n" tokens? I suppose separate tokens like that won't be seen in the input string, so they'll never be predicted?

Re: We Found an Neuron in GPT-2

#105

Post by zeven7 »

justusthane wrote: post #100
furyofantares wrote: post #98

Earlier quoted context omitted.

I never know what I'm about to say, but somehow coherent sentences come out. I certainly don't know what words a sentence is going to end with when I'm thinking or saying the first words in the sentence. I just think or say the sentence from start to finish, never knowing what the next word is going to be as I'm thinking the current one, and by the end of it I've thought or said a full sentence that makes sense.

This is so weird. I am very “In my head” and have generally planned what I’m going to say before I open my mouth. It makes me feel like I’m not very good at conversation though, especially smalltalk.

Yeah, I'm like you but definitely know people like parent. I never really fully realized that's what they were doing though. I think I assumed they were way faster at planning their sentences than I was. Now that I know this is an option, I'm interested in giving it a try.

Re: We Found an Neuron in GPT-2

#106

Post by furyofantares »

zeven7 wrote: post #105
justusthane wrote: post #100

Earlier quoted context omitted.

This is so weird. I am very “In my head” and have generally planned what I’m going to say before I open my mouth. It makes me feel like I’m not very good at conversation though, especially smalltalk.

Yeah, I'm like you but definitely know people like parent. I never really fully realized that's what they were doing though. I think I assumed they were way faster at planning their sentences than I was. Now that I know this is an option, I'm interested in giving it a try.

I'm actually very quiet because I do a lot of planning before I speak. The planning comes in complete sentences, one word at a time, though.

Re: We Found an Neuron in GPT-2

#107

Post by sitkack »

Baeocystin wrote: post #58
LudwigNagasena wrote: post #13

It’s notable how successful LLMs despite the lack of any linguistic tools in their architectures. It would be interesting to know how different a model would be if it operated on eg dependency trees instead of the linear list of tokens. Surely, the question of “a/an” would be solved with ease as the model would be required to come up with a noun token before choosing its determiner. I wonder if the developers of LLMs…

Approaches such as you describe have been the dominant method for decades. That we finally 'cracked' natural language generation with tools that literally encode nothing about grammar ahead of time is one hell of a lesson, early days as it is in the learning of it.

Ref, A Bitter Lesson

Re: We Found an Neuron in GPT-2

#108

Post by tkanarsky »

mungoman2 wrote: post #89
rootusrootus wrote: post #83

That's interesting. I just asked ChatGPT to explain how it decides "a" vs "an" and it confirmed that it will retroactively change "a" to "an" when it finds that the following word sounds like it starts with a vowel sound. It could have been hallucinating, of course. But it does seem like it occasionally alters already generated words as it goes, at least I think I've seen it do that.

ChatGPT can't really introspect though. It has no idea how it works, so it'll just blurt out something that sounds feasible, biased by your prompt. The slow progression of ChatGPT output is just a property of the output layer. The language engine doesn't work slowly like that, and once a token has been generated it can't backtrack.

What specifically causes the output layer to trickle-print the response like that? I thought it was a skeumorphic effect to simulate a human typing out an answer slowly.

Re: We Found an Neuron in GPT-2

#109

Post by ly3xqhl8g9 »

clementneo wrote: post #75

Co-author here! I'm kind of surprised that this made it to the top of HN! This was a project in which Joseph and I tried to reverse engineer the mechanism in which GPT-2 predicts the word 'an'. It's crazy that large language models work so well just by being trained as a next-word-prediction model over a large amount of text data. We know how image models learn extract the features of an image through convolution[1],…

Could a "type system" for neural weights be developed? Given a self-driving system, to be able to statically check that the neurons have the "Person" type, the "Don't Run Over Person" type, and so forth. What happens if you "transplant" the weights for ' an' to another network, some kind of transfer learning but componentized, does it still predict as accurately? If neural networks could be assembled from "types" it would be much easier to trust them.

Re: We Found an Neuron in GPT-2

#110

Post by adlpz »

jdougan wrote: post #95
SamBam wrote: post #80

Interestingly I feel like humans have this as well, sometimes. Sometimes if someone is working though a complex thought and they're not really sure where they're going, they'll pause while thinking of the word they want to use, and might sound like "the discussion is an... an... epistemological one" Obviously they may have been conscious that the next word was going to start with "epi..." and they are just trying to…

> "the discussion is an... an... epistemological one" I hear that in the voice of Agent Smith, from The Matrix.

Quite an appropriate reference when talking about neurons on an artificial intelligence :)
Post reply on HN