61–70 of 392 posts
[flagged]
https://dl.acm.org/doi/10.1145/3442188.3445922
"On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?" by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^H^HShmargaret Shmitchell
[flagged]
A good article and well articulated! I would change the introduction to be more impartial and not anthropomorphize GPT. It is not smart and it is not skilled in any tasks other than that for which it is designed. I have the same reservations about the conclusion. The whole middle of the article is good. But to then compare the richness of our human experience to an algorithm that was plainly explained? And then to sp…
I'm planning on continuing this vulgarization series of "Simply explained", for instance to cover how computers communicate, keep an eye out for them! Regarding the speculation/destroy society, I was directly answering questions that I got from laypeople around me. The consequences on society I don't think are much speculation: it's going to have a big effect on many jobs, just like AI has started to have but much mo…
I will, thank you! :)
> Regarding the speculation/destroy society, I was directly answering questions that I got from laypeople around me.
I get that. I think it's important in these times that we educate laypersons rather than froth up fears about "AI". It doesn't help, I suppose, that we get questions like this because some lazy billionaire decided to run their mouth off about this or that. Which society then treats like it is news and established fact.
I don't think the speculation about consciousness is as well informed as the rest of the article. There is plenty of science and research about it available and its definition extends well beyond humans! Our understanding of what consciousness is is a thoroughly researched topic in psychology, physiology, biology, etc! It's a fascinating area of study.
Best of luck and keep up the good work!
This is confusing, using the semantic vectors arithmetic of embeddings is not very relevant to transformers and its completely missing the word 'attention'. I don't think transformers are that difficult to explain to people , but it is hard to explain "why" they work. But i think it's important for everyone to look under the hood and know that there are no demons underneath.
It's a bit more plausible when we phrase it that way...
What I wonder most is how it encodes knowledge/state other than in the sequence of queries/responses. Does it not have a "mind"? If I play a number guessing game, can I tell it to "think of a number between 0 and 100" and then tell me if the secret number is higher/lower than my guess (For a sequence of N guesses where it can concistently remember it's original number)? If not, why? Because it doesn't have context? I…
The thing to remember is that GPT has no state apart from the context, so it can't "remember" anything apart from what's in the text. That doesn't mean it shouldn't be able to stay consistent in a guessing game but it does mean it can't keep secrets.
Some of that can be solved with layers above GPT where say it it told it can save "state" that isn't passed on to the human but fed back in to generate the next response. But the size of that context is very limited. (a few thousand words)
There seem to be a fair number of experiments playing with giving GPT this kind of long term memory, having it establish goals then calling it over and over as it accomplishes subgoals to try to work around those limitations.
What I wonder most is how it encodes knowledge/state other than in the sequence of queries/responses. Does it not have a "mind"? If I play a number guessing game, can I tell it to "think of a number between 0 and 100" and then tell me if the secret number is higher/lower than my guess (For a sequence of N guesses where it can concistently remember it's original number)? If not, why? Because it doesn't have context? I…
In other words, it's seen (via its extremely large training set) that when asked that specific question, the response is most often a character from a particular set of characters, which happens to represent the numbers 0 through 100. It doesn't "understand" what that means in any real way. If the internet was full of examples of people answering "monkey" to that question, that is what it would sometimes answer.
Earlier quoted context omitted.
Embeddings and their relationship to each other are definitely relevant to transformers. Why do you think that's not the case?
gptX embeddings aren't even words. Even so, the embedding relationship is useful but not the core of what transformers do to find relationships between words in sequences.
I agree with the sentiment that each individual dimension isn't meaningful, and I also feel like it's misleading for the article to frame it that way. But there's a grain of truth: the last step to predicting the output token is to take the dot product between some embedding and all the possible tokens' embeddings (we can interpret the last layer as just a table of token embeddings). Taking dot products in this space are equivalent to comparing the "distance" between the model's proposal and each possible output token. In that space, words like "apple" and "banana" are closer together than they are to "rotisserie chicken," so there is some coarse structure there.
Doing this, we gave the space meaning by the fact that cosine similarity is meaningful proxy for semantic similarity. Individual dimensions aren't meaningful, but distance in this space is.
A stronger article would attempt to replicate the word2vec analogy experiments (imo one of the more fascinating parts of that paper) with GPT's embeddings. I'd love to see if that property holds.
Earlier quoted context omitted.
What gives you any confidence that the way GPT4 comes up with answers is qualitatively different from humans? Why should the emulation of human though, a result of unguided evolution, require anything more than properly wired silicon?
That's highly reductive of our capacities. We are not weighted transformers that can be explained in an arxiv paper. GPT, at the end of the day, is a statistical inference model. That's it. It's not going to wake up one day, decide it prefers eggs benny and has had enough of your idle chatter because of that sarcastic remark you made last week. Could we simulate a plausibly realistic human brain on silicon someday? I…
I'm not saying that GPT4 is as capable as a human-- it can not be, by design, because its architecture lacks memory/feedback paths that we have.
What I'm saying is that HOW it thinks might already be quite close in essence to how WE think.
> We are not weighted transformers that can be explained in an arxiv paper. GPT, at the end of the day, is a statistical inference model. That's it.
That is true but uninteresting-- my counterpoint is: If you concede that our brain is "simulatable", then you basically ALREADY reduced yourself to a register based VM-- the only remaining question is: what ressources (cycles/memory) are required to emulate human thought in real time, and what is the "simplest" program to achieve it (that might be something not MUCH more complicated than GPT4!).
Earlier quoted context omitted.
The more constraints there are (e.g. like your example) the better it should perform. So it disappoints me when copilot, knowing what libraries are available in the IDE it's running in, hallucinates up a method call that doesn't exist. Separately (and apologies for going on a tangent), where do you think we are in the Gartner cycle? Around GPT3 time I was expecting for trough of disillusionment to come, particularly…
> hallucinates up a method call that doesn't exist I actually think it helps to reframe this. It hallucinates up a method call that predictively should exist. If you're working with boto3, maybe that's not actually practical. But if it's a method within your codebase, it's actually a helpful suggestion! And if you prompt it with the declaration and signature of the new method, very often it will write the new helper…
What I wonder most is how it encodes knowledge/state other than in the sequence of queries/responses. Does it not have a "mind"? If I play a number guessing game, can I tell it to "think of a number between 0 and 100" and then tell me if the secret number is higher/lower than my guess (For a sequence of N guesses where it can concistently remember it's original number)? If not, why? Because it doesn't have context? I…
Me: I want to play a game with you, you have a number from 1-10 in your mind. I try to get the number. If I’m right you tell me that I’m right, I’m too low you tell me so, if I’m too high you tell me too high. Let’s play!
GPT: All right, I have a number between 1 and 10 in my mind. Go ahead and guess!
Me: 2
GPT: Too low! Try again.
Me: 9
GPT: Too high! Try again.
Me: 5
GPT: You got it! The number I had in mind was 5. Well done!