We Found an Neuron in GPT-2
clementneo.com
We Found an Neuron in GPT-2
1–10 of 177 posts
Re: We Found an Neuron in GPT-2
#2[dead]
Re: We Found an Neuron in GPT-2
#3> The choice depends on whether the word that comes after starts with a vowel or not
Or an aitch / h, but only sometimes.
Re: We Found an Neuron in GPT-2
#4> The choice depends on whether the word that comes after starts with a vowel or not Or an aitch / h, but only sometimes.
More accurately, it's whether the word that comes after starts with a vowel sound. This is why `an 'istoric` is correct, and `a historic` is correct, but `an historic` is incorrect (as famously used by Steven Colbert).
Re: We Found an Neuron in GPT-2
#5[deleted]
Re: We Found an Neuron in GPT-2
#6I wonder if this stuff will ever be applicable to a person and a laptop (or if it is now?).
Ie this seems like such a cool area to be in but the data volumes required are huge, complex, etc. Code is simple, cheap, lean, etc by comparison.
Do we have any insight on how this area of research could be usable with less hardware and data? Is there a visible future where a guy and a laptop can make a big program? (without depending on tech getting small/cheap in 50 years or w/e)
Re: We Found an Neuron in GPT-2
#7The way they're going about this investigation is reminiscent of how we figure things out in biology and neuroscience. Perhaps a biologist won't do the best job at fixing a radio [1], but they might do alright debugging neural networks.
1: https://www.cell.com/cancer-cell/pdf/S1535-6108(02)00133-2.p...
Re: We Found an Neuron in GPT-2
#8Since complex systems are composed of simpler systems, seems like for any sufficiently complex system you'd be able to find subsets of it which are isomorphic which any sufficiently simple system.
Re: We Found an Neuron in GPT-2
#9I wonder if this stuff will ever be applicable to a person and a laptop (or if it is now?). Ie this seems like such a cool area to be in but the data volumes required are huge, complex, etc. Code is simple, cheap, lean, etc by comparison. Do we have any insight on how this area of research could be usable with less hardware and data? Is there a visible future where a guy and a laptop can make a big program? (without…
You could run this analysis on a laptop very easily.
The pile-10k dataset they used for analysis is 33MB, and GPT2 runs ok on a CPU. For the full 10K analysis it's probably quicker to get a GPU though.
Re: We Found an Neuron in GPT-2
#10> We started out with the question: How does GPT-2 know when to use the word an over a? The choice depends on whether the word that comes after starts with a vowel or not, but GPT-2 is only capable of predicting one word at a time. We still don’t have a full answer...
I'm not sure I understand why this is an open question. While I get that GPT-2 is predicting only one word at a time, it doesn't seem that surprising that there might be cases where there is a dominant bigram (ie "an apple" in the case of their example prompt) that would trigger an "an" prediction, without actually predicting the following word first.
Am I missing something?