Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

81–90 of 231 posts

Re: And yet It Understands

#81

Post by PoignardAzur »

raincole wrote: post #73
PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

I tried it several (>10) times and I couldn't get the input suggestions. https://i.imgur.com/SaVjSzT.png Possibility 1: It's an extremely rare case and I need to try more Possibility 2: MS patched it quickly when they saw this article Possibility 3: The author lied for page views Choose your poison.

> Possibility 2: MS patched it quickly when they saw this article

I'd guess Possibility 2.5: MS made other patches that stopped the behavior, not because they saw the article, but because they're constantly patching the model.

Re: And yet It Understands

#82

Post by ivxvm »

glenstein wrote: post #71
MrScruff wrote: post #37

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human.…

>LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat. Just as an example to illustrate your point, yesterday saw a Twitter meme that had multiple overlapping Venn diagrams, where Chicago was not only in a Venn diagram for a type of deep dish pizza, but is also a city, and is also a play, and also a format for…

Maybe it simply doesn't understand Venn diagrams well - after all, it's a visual concept.

Re: And yet It Understands

#83

Post by glenstein »

doctor_eval wrote: post #51
glenstein wrote: post #29

Earlier quoted context omitted.

Right, over the totality of things that it reasons about, to some degree it will make inroads to correctly answering these kinds of questions, and in some ways it'll make errors, and what's interesting, is it'll make errors because the way in which it's attempting to answer them bears a lot of the hallmarks that we associate with conceptual understanding, rather than the mechanical operations of a calculator which tr…

Yes, exactly this. I recently asked ChatGPT to write me some Go code. What it wrote was mostly fine, but it incorrectly used a method invocation on an object, instead of the correct code which was to pass the object to a function at the package level (aka static method). I think it's a real stretch to suggest that this could happen as a result of simply regurgitating strings it's seen before, because the string it em…

Just on the point of forgetting, to pass the time while bored at work I told Chad GPT about my fantasy baseball team, and over the course of the conversation it forgot who my first baseman was but input the name of an entirely different first baseman who I had never mentioned, but who is a real person and who in fact was similar in a lot of ways to my actual first baseman. And sometimes when it would attempt to recall my lineup it would seem to 'forget' certain players even if it remembered them later.

Re: And yet It Understands

#84

Post by famouswaffles »

FartyMcFarter wrote: post #31
PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

Before we get too excited, does anyone know how the input suggestions are generated?

They are generated by the same model. At least, you can ask Sydney/Bing to alter the suggestions and it works.

Re: And yet It Understands

#85

Post by espadrine »

the_gipsy wrote: post #69
MrScruff wrote: post #37

This article lines up well with my feelings on the matter. In general, people seem to understate the emergent behaviours of ML models, while overstating the uniqueness of human intelligence. I think a lot of this is down to the fact that although both systems exhibit a form of intelligence, they’re very different. LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human.…

Maybe it is human like intelligence already. Maybe our internal monologue is just a better trained and refined ChatGPT. And maybe that is all the magic that is necessary for this holy grail of consciousness, there is no quantum brain, no nothing. Just a stream of the next word that says that we are there, therefore we are. That is what scares me.

> Maybe it is human like intelligence already.

It objectively is not.

First, intelligence (human or otherwise) is typically defined as the sum of value functions of a given policy across all states, divided by an exponential of the complexity of the environment[0] (page 23). The GPT-4 technical report[1] specifically points out that it is unable to make money when given an initial amount and AWS access (appendix H § 2.9). A human can.

Second, let's look at it mechanically. It is very easy to anthropomorphize machines, but they are just built differently. Neurons fundamentally are not approximated by self-attention and it is highly unlikely that they improve through backpropagation. In terms of behaviour, we can and do notice differences. Transformers can, and do, make logical mistakes; when asked to immediately answer a complex question, they can only think through it for a fixed amount of time, and thus fail to give an accurate answer. More importantly, they just don't care about their (or your) own survival; they can give lethal advice. All it takes is to find a prompt for which the most common completion is lethal, and they will just complete that. Removing those completions is a whack-a-mole in RLHF: each that gets popular on Twitter will be fixed, but there are more, and GPT just won't take extra time to think just because a life is on the line.

[0]: https://arxiv.org/pdf/0712.3329.pdf

[1]: https://cdn.openai.com/papers/gpt-4.pdf

Re: And yet It Understands

#86

Post by Certhas »

Human intelligence evolved with the goal to survive and procreate. GPT intelligence evolved to mimick human speech.

Both tasks require a conceptual understanding of the world humans inhabit, but otherwise the two tasks that gave rise to these intelligences are utterly different. We should expect these intelligences to be completely different.

Re: And yet It Understands

#87

Post by civilized »

ccppurcell wrote: post #8

I just asked chatgpt whether 3442177452 is prime. It insisted that 58657 is a factor (it's not) on the basis that it's the largest prime less than or equal to the square root (which I think is correct but irrelevant), and even though it gave a non zero remainder when dividing the two numbers (I did not check if the remainder is correct). Then it gave a (wrong) factorisation, not even using 58657. It's cool and it wil…

Using your number as a jumping off point, I went down a very entertaining rabbit hole with ChatGPT just now. I will not paste the whole dialogue here, but I would like to assure everyone that I made no attempt to mislead ChatGPT in any way. I simply attempted to draw out its knowledge, and questioned it Socratically along the way. Select responses are quoted below.

The first thing I did was ask it about the prime factorization of 3,442,177,452. It said it was 2^2 * 863 * 198853. Then I asked it to compute 4 * 863 * 198853. It said:

> the product of 2^2 * 863 * 198853 is equal to 3,442,177,452, while the product of 4 * 863 * 198853 is equal to 689,401,672.

I then got it to note that 2^2 = 4, and questioned it about what this implies about its previous statements. After a couple attempts, it did figure out the results of the two multiplications should be the same, and that therefore there was a mistake in its previous responses. It then claimed that the results of both multiplications should be 3,442,177,452. Then I asked it how it derived the answer. It said:

> I performed the multiplication of 4 * 863 * 198853 using the multiplication algorithm that is part of my programming. This algorithm uses basic arithmetic operations to calculate the product of the three numbers, resulting in the correct answer of 3,442,177,452.

Then I told it that I got 4 * 863 * 198853 = 686,440,556 from Google's calculator. At this point, it disclaimed all previous results. I then asked it why it had gotten incorrect answers, and got this astonishing response:

> The reason for the error is that I incorrectly calculated the product of 4 * 863 * 198853 by mistakenly assuming that 2^2 is equal to 2, rather than 4.

> In my first response, I correctly calculated the product of 2^2 * 863 * 198853 to be 3,442,177,452. However, in my subsequent responses, I made a mistake by stating that the product of 4 * 863 * 198853 is equal to 3,442,177,452, which is incorrect. Instead, the correct product of 4 * 863 * 198853 is 686,440,556.

Recall that I had previously gotten it to agree that 2^2 * 863 * 198853 and 4 * 863 * 198853 were equal because 2^2 = 4. Here it immediately goes back on that in a slapdash attempt to explain its previous mistakes.

What I find interesting about this whole AI discourse is that no one seems to bother with this kind of test, one where we take the hype about GPT's "general intelligence" seriously and actually test the limits of that, with true respect for the concept of "general intelligence". Honestly, there seems to be a sort of soft bigotry of low expectations. Everyone wants to give the AI problems that it can solve and pat it on the head for its successes. There is less interest in finding ways to probe whether it is truly as "generally intelligent" as the hype says.

Re: And yet It Understands

#88

Post by glenstein »

zetalyrae wrote: post #38
smitty1e wrote: post #24

> But nobody knows how GPT works. They know how it was trained, because the training scheme was designed by humans, but the algorithm that is executed during inference was not intelligently designed but evolved, and it is implicit in the structure of the network, and interpretability has yet to mature to the point where we can draw a symbolic, abstract, human-readable program out of a sea of weights. I object. ChatGP…

I don't see the relevance. Natural intelligence has not been understood because if our brains were simple enough that we could understand them, we would be so simple we couldn't. This explains why it has not been reproduced. >There is no repeatable experiment going from inorganic matter to self-aware, self-replicating life. You can do the RNA world experiments in a lab, it would just take a lot of time and a lot of p…

>Natural intelligence has not been understood because if our brains were simple enough that we could understand them, we would be so simple we couldn't. This explains why it has not been reproduced

I don't believe that it works this way, for two reasons. One, the logic would seem to work the other way as well, if our brains are so sophisticated that they make natural intelligence possible, then we should be so intelligent that we would have the means to understand them. (At the end of the day I think this is a case where analogies aren't good enough to settle it one way or the other.)

But secondly, the overall architecture of the brain can be understood in terms of underlying principles that are reapplied over and over, and so we can conceivably 'compress' the totality of information about the brain's architecture to the principles that explain why it functions. Granted we haven't done that yet and that could be extremely difficult, but, I don't think it's forbidden by necessity or by some transcendent principle. I would say that I agree both with your reply about RNA experiments being feasible, and I share your confusion as to how any of that was relevant.

Re: And yet It Understands

#89

Post by glenstein »

ivxvm wrote: post #82
glenstein wrote: post #71

Earlier quoted context omitted.

>LLMs deliver mastery of natural language that would normally be a signal for a highly intelligent human. While in other ways they’re less intelligent than a cat. Just as an example to illustrate your point, yesterday saw a Twitter meme that had multiple overlapping Venn diagrams, where Chicago was not only in a Venn diagram for a type of deep dish pizza, but is also a city, and is also a play, and also a format for…

Maybe it simply doesn't understand Venn diagrams well - after all, it's a visual concept.

My thinking is that it relates to organizing concepts into categories, which can challenge a system to prove how deep it is capable of going into abstract associations, and searching through a full range of possible abstract associations.

I suspect ChatGPT probably can be reasonably effective at understanding, or pretending to understand numerous other visual concepts.

Re: And yet It Understands

#90

Post by FartyMcFarter »

famouswaffles wrote: post #84
FartyMcFarter wrote: post #31

Earlier quoted context omitted.

Before we get too excited, does anyone know how the input suggestions are generated?

They are generated by the same model. At least, you can ask Sydney/Bing to alter the suggestions and it works.

That sounds likely. I'm also wondering whether they're generated with extra hidden prompts, and whether they're generated independently or as a sequence that depends on the other input suggestions. Without knowing that, it's hard to evaluate how surprising any particular set of input suggestions is.
Post reply on HN