Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

61–70 of 231 posts

Re: And yet It Understands

#61

Post by glenstein »

ben_w wrote: post #28
glenstein wrote: post #16

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon. Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a prof…

Interesting insight. For me, the (tree) analogy is that of an illusion of progress: while the treetop is closer than the ground, there is no tree that can ever be tall enough to get the rest of the way. So, it isn't supposed to be used to say "you can't do it at all", just "you can't do it like that ". But metaphors get mixed as soon as they leave the bottle of text and enter the cocktail shaker of other people's min…

>So, it isn't supposed to be used to say "you can't do it at all", just "you can't do it like that".

And I'm of two minds on this, because I find that to be a respectable position, and in fact, for this reason, I hope you show up to continue debates on this because your voice would be helpful and counteracting what I believe is the less reasonable position. Depending on how long you've been following debates on the topic, which have raged from essentially the 1950s through now, I would guesstimate that "you literally can never get to the moon" has at least at times enjoyed dominance as the majority position, and when not the majority position, is at least credited as being a respectable one.

And even in this dressed up reasonable version, it still feels wrong to me in an important way: if you did have a tree that was 328,900 miles tall, you could reach the Moon. There are so many opportunities along the way to mentally short circuit, and slip into practical considerations and lose sight of the principle. Of course no such tree exists, but it has critically illustrated that the entirety of the space from here to there is in principle traversable, and it's the insight into this principle that ultimately will get us from here to there.

It does mean you need to shift your focus of research from arborism to jet propulsion, and that is the important point made by your version of this argument. But it amounts to joining the 'arborists' in championing the possibility of getting there rather than being a skeptic of the possibility, and I feel like most people making this point imagine themselves to be taking the sides of the skeptic.

Re: And yet It Understands

#62

Post by _dain_ »

foldr wrote: post #44
zetalyrae wrote: post #40

Earlier quoted context omitted.

The point is that at no point in evolutionary history did a cognitive scientist sit down and write a bunch of S-expression GOFAI rules for human cognition.

Of course not, but who is claiming that this happened? I'm not sure I can think of even single person who is both (i) an intelligent design advocate and (ii) explicitly committed to a symbolic model of human cognitive capacities. This seems like a straw man. You might as well say that at no point in evolutionary history did an embryologist ever write down a plan for the eight developmental stages of a human fetus.

the "intelligent design advocates" doesn't refer to creationists in this context, it refers to GOFAI people.

Re: And yet It Understands

#63

Post by doctor_eval »

smitty1e wrote: post #24

> But nobody knows how GPT works. They know how it was trained, because the training scheme was designed by humans, but the algorithm that is executed during inference was not intelligently designed but evolved, and it is implicit in the structure of the network, and interpretability has yet to mature to the point where we can draw a symbolic, abstract, human-readable program out of a sea of weights. I object. ChatGP…

So what if it's electrical signals. It's not really 1s and 0s is it, they are grouped together into floating point numbers, right? And at some point there are enough ones and zeroes that you can effectively simulate just about any analog state. So why should we assume it's not possible to simulate the states of neurons? And why do we assume that the brain's structure is the only way to produce intelligence? Surely - like other biological systems - they are a specific implementation of what nature allows, but not necessarily the only one.

> nowhere near capable of doing the impossible, e.g. predicting the weather with fidelity substantially into the future

Why is it that people keep telling us GPT isn't capable of being conscious or having understanding because it's unable to perform tasks that individual humans can't do?

Re: And yet It Understands

#64

Post by PoignardAzur »

bmacho wrote: post #19
PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

Wait, are those supposed to be input suggestions, like you click on them and it pastes them in? Sydney is not supposed to give coherent 3-part messages using them, right?

> Sydney is not supposed to give coherent 3-part messages using them, right?

Right, that's the "what the actual fuck" part.

This raises some very interesting questions about how Sydney generates its output and the input suggestion. Presumably the LLM is given a prompt like "First generate an answer to the previous text, then generate three input suggestions for the user"; also, the fact that Sydney "hides" the messages in input suggestions suggests that it's aware the main message is "censored", which seems really surprising. As in, this was the kind of scenario that AI safety skeptics would dismiss with "of course it's not going to be implemented that way"-type assertions.

So this seems like evidence that not only Sydney is "told" to generate both the answer and the prompt suggestions, but it's also being "told" to censor the answer (as opposed to the answer just being replaced with a placeholder text after the fact), and for some reason it "decides" to evade the censorship by passing additional info in the suggestions.

(And yes, AI rigorists will tell me that it isn't actually "told" anything and it doesn't "decide" anything; it's just a prediction engine that predicts what an user with the "Sydney" personality would say in the given context. But the things it ends up predicting seem pretty fucking agent-like.)

It's always possible we're overblowing things, of course. But this seems to me like the first example of a LLM not just being misaligned, but actively exploiting a loophole in its surface-level alignment to accomplish some deeper goals. Alarming.

Re: And yet It Understands

#65

Post by glenstein »

raincole wrote: post #42
glenstein wrote: post #16

>I was a deep learning skeptic. I doubted that you could get to intelligence by matrix multiplication for the same reason you can’t get to the Moon by piling up chairs I've always been fascinated by this example. I've also heard it referred to as climbing a tree won't get you to the Moon. Because, for some reason, people think that's an argument against the possibility of getting to the Moon when it's actually a prof…

I don't know. I've been using Copilot, ChatGPT, and Bing Chat intensively in the past month. So far I still think the metaphor "you can’t get to the Moon by piling up chairs" aged very well. > A criticism that would make more sense would be something along the lines of "piling up colors you won't get you any closer to the Moon", since colors aren't even the right kind of thing, and you can't aggregate them in a way t…

>means that even chairs and rockets share some attributes (helping you get higher), the difference between them is still qualitative, not quantitve.

I don't think so, because the needed 'quality' is the ability to traverse space. So I don't think I agree that the qualitative piece is missing.

Perhaps the moon example is helpful here because the real solution, a rocket ship, uses propulsion rather than sheer mass, and so you could say that some function such as propulsion is 'qualitatively' lacking. But even that I believe exhibits the very form of confusion that I'm criticizing, which I'll explain below.

>The problem isn't "we don't have enough chairs."

Well, in a way it is. And at the risk of sounding like the type of question that Randall Monroe would answer in the book What If, even this most vulgar example would in a literal case prove true. Given enough chairs, they would topple over, but create a pile that overtime would spill and evenly distribute over the earth, and it would be the whole earth that grows until it's size is so large that it's close to the Moon.

And, I wasn't even trying to make this point, but it's actually kind of a perfect example here because Chat GPT has made all of its progress by throwing chairs (data) at the problem.

Re: And yet It Understands

#66

Post by barking_biscuit »

foldr wrote: post #12
Etheryte wrote: post #11

Earlier quoted context omitted.

This is a common misconception. ChatGPT is not supposed to be good at this — it's a language model, not a maths model or data science model or whatnot. This is exactly why they have plugins, such as the one for Wolfram Alpha.

Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

>Of course people who actually know how ChatGPT works don't expect it to be able to magically solve mathematical problems. However, these examples do show that ChatGPT isn't (contrary to some of the hype) deriving a deep conceptual understanding of its input data.

I don't think it really makes the case that you think it makes. Humans learn language and concepts through sentences, and in most cases semantic understanding can be built up just fine this way. It doesn't work quite the same way for math. When I look at the numbers in the example, I have no idea if they are prime or factors because they themselves don't have much semantic content. In order to understand whether they are those things or not actually requires to stop and perform some specific analysis on them learned through internalizing sets of rules that were acquired through a specialized learning process. Humans themselves don't learn math by just talking to one another about it, rather they actually have to do it in order to internalize it.

I suspect it's simply a case of we haven't sent ChatGPT to highschool yet. There will be some particular trick in terms of training methodology that trains the network to perform that more specialized analysis that doesn't simply emerge out of 'predict the next token from all this random internet text' as it's likely severely underrepresented in the training data in the first place, but I think it ought to be perfectly possible.

Re: And yet It Understands

#67

Post by MrScruff »

stereolambda wrote: post #56

I sense emotional and identity-based thinking sneaking in both this article and many of its stated adversaries. Yes, anti-GPT punditry is getting ridiculous, but on the other hand, it's important to examine what is happening through scientific-minded and skeptic lens. The alternative is jumping at every symptom that could be caused by a "personality" existing inside a model, but could also be a combination of chance…

Agreed, and this is where there needs to be a line drawn. GPT is trained to emulate the patterns in the text it was trained in, and it is very likely learning higher level relations between concepts/states where it needs to in order to make better predictions. And this is amazing in of itself.

But that doesn’t mean it ‘feels emotions’ related to these concepts because it hasn’t had the billions of years of reinforcement learning that we have to tell us some of these concepts should induce fear/desire etc.

I have no doubt that AGI is possible but I really don’t expect the intelligence that results to resemble human intelligence. I would expect dolphin intelligence to be more ‘similar’ to human intelligence since at least we have a common ancestor.

Re: And yet It Understands

#68

Post by doctor_eval »

rain1 wrote: post #54
doctor_eval wrote: post #35

Earlier quoted context omitted.

And what I think we'll find is that we are, essentially, stochastic parrots. (I'm OK with that. It is what it is).

sometimes. :) sometimes we are so much more.

I often think about this problem and I keep returning to the thought that maybe we're close to understanding how consciousness works, maybe these LLMs are actually getting us closer to understanding this thing.

But some people are going to be disappointed because it will remove all doubt about how un-special humans are. We're just a bunch of neurons, which are made out of physics.

But I'm not disappointed. This stochastic parrot is amazed! Nature has created a system which is able to understand itself! That's absolutely incredible.

Re: And yet It Understands

#69

Post by the_gipsy »

MrScruff wrote: post #37

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human.…

Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.

Re: And yet It Understands

#70

Post by maister »

tjr wrote: post #45

I keep being reminded of Paul Graham's "plan for spam", in that he devised a simple statistical evaluator, and was surprised that it worked so well to distinguish ham from spam. These AI tools have been trained on a great deal of written language artifacts and exhibit a surprising level of what appears to be concept understanding. Perhaps the real surprise is that language conveys concepts better than we previously t…

> the real surprise is that language conveys concepts better than we previously thought?

Is it really that much of a surprise? Isn't the whole purpose of language to transport concepts?

I mean, our brains are not directly connected to each other, yet you just transferred a concept (which was a result of your thinking and understanding) to my brain by using language.

Post reply on HN