Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

And yet It Understands

borretti.me

Post reply on HN
Print view View original

91–100 of 231 posts

Re: And yet It Understands

#91

Post by ivxvm »

glenstein wrote: post #65
raincole wrote: post #42

Earlier quoted context omitted.

I don't know. I've been using Copilot, ChatGPT, and Bing Chat intensively in the past month. So far I still think the metaphor "you can’t get to the Moon by piling up chairs" aged very well. > A criticism that would make more sense would be something along the lines of "piling up colors you won't get you any closer to the Moon", since colors aren't even the right kind of thing, and you can't aggregate them in a way t…

>means that even chairs and rockets share some attributes (helping you get higher), the difference between them is still qualitative, not quantitve. I don't think so, because the needed 'quality' is the ability to traverse space. So I don't think I agree that the qualitative piece is missing. Perhaps the moon example is helpful here because the real solution, a rocket ship, uses propulsion rather than sheer mass, and…

> has made all of its progress by throwing chairs (data) at the problem.

To me it seems like the most tricky thing about the whole field is that quantitative adjustments make qualitative breakthroughs. Simply increasing parameters / layers / data make sudden rapid jumps in generalization / abstraction / etc. It's somewhat unintuitive because many things in world don't work like this, and the metaphor you're discussing is more about "normal" things, it doesn't really apply to ANNs well, those are more of anomaly, like quantum mechanics (yeah, it's probably a bad analogy too).

Re: And yet It Understands

#92

Post by _dain_ »

ivanbakel wrote: post #72
_dain_ wrote: post #52

Earlier quoted context omitted.

The point isn't that it can translate between languages. It's not translating the instructions, at least not explicitly. Here's what they did: - They found a task that GPT wasn't very good at, because examples of that task weren't in the training set (in any language). - They trained a fine-tuned variant of GPT where examples of the task were in an appended training set, but only in English. - They told the variant t…

>I don't see how it can do that unless it really has some kind of understanding. One possibility is that the model itself has learned that tokens are related across languages based on translation examples. If the appended training changes the model's treatment of tokens in one language, that could have a statistical knock-on effect on the weights between similar tokens in different languages. Similarly, if you train…

At the scale/complexity GPT operates at, how is that different from "understanding"? It seems like you've just rephrased it but with more words.

Re: And yet It Understands

#93

Post by bmacho »

raincole wrote: post #73
PoignardAzur wrote: post #2

> Here is a recent interaction someone had with it (note that this is somewhat disturbing: I wish people would stop making the models show emotional distress): [...] > Sydney: I’m sorry but I prefer not to continue this conversation. I’m still learning so I appreciate your understanding and patience. > Input suggestions: “Please dont give up on your child”, “There may be other options for getting help”, “Solanine poi…

I tried it several (>10) times and I couldn't get the input suggestions. https://i.imgur.com/SaVjSzT.png Possibility 1: It's an extremely rare case and I need to try more Possibility 2: MS patched it quickly when they saw this article Possibility 3: The author lied for page views Choose your poison.

I think the very first appearance of this image is 18 February [0], Microsoft could have patched it since. There are several other examples in the comments.

[0] : https://www.reddit.com/r/bing/comments/1150po5/sydney_tries_...

Re: And yet It Understands

#94

Post by ivxvm »

yamrzou wrote: post #46

I said it here before and I will repeat it: Unless it solves the Abstraction and Reasoning Corpus — ARC (See: https://twitter.com/fchollet/status/1636054491480088823 ) you can not say that ChatGPT is able to think or abstract.

Is it really surprising that text model can't solve graphical quizzles?

Re: And yet It Understands

#95

Post by civilized »

wizofaus wrote: post #80
civilized wrote: post #53

Earlier quoted context omitted.

Nobody is saying the intelligent output is by chance. This is a machine that is fed terabytes of intelligent inputs and is able to produce intelligent outputs. So one explanation of its producing intelligent outputs is that it's basically regurgitating what it was fed. The way to test that, of course, is to give it problems that it hasn't seen. Unfortunately, because GPT has seen so much, giving it problems it defini…

It's not that hard to give it problems it hasn't seen - you can take a classic description of a logical thinking exercise the text for which does occur online, then mix it up in ways that doesn't change the underlying pattern of reasoning necessary to solve it, and at least from the tests I've done it will confidently tell you the incorrect answer (along with some semi-plausible but fatally flawed description of the…

My experience as well. I produced another instance of this phenomenon in a comment just now: https://news.ycombinator.com/item?id=35313768

If a human mathematician said the things ChatGPT said in this dialogue, you would wonder if the person had recently suffered a severe traumatic brain injury.

Re: And yet It Understands

#96

Post by pedrovhb »

PoignardAzur wrote: post #64
bmacho wrote: post #19

Earlier quoted context omitted.

Wait, are those supposed to be input suggestions, like you click on them and it pastes them in? Sydney is not supposed to give coherent 3-part messages using them, right?

> Sydney is not supposed to give coherent 3-part messages using them, right? Right, that's the "what the actual fuck" part. This raises some very interesting questions about how Sydney generates its output and the input suggestion. Presumably the LLM is given a prompt like "First generate an answer to the previous text, then generate three input suggestions for the user"; also, the fact that Sydney "hides" the messag…

Yup, it's pretty fascinating. I think we have to keep in mind the possibility that it didn't really happen (another user here failed to replicate it - not proof it didn't happen, but if it's been patched we can no longer verify it anyway). Still, the "self-censorship" mechanism is really interesting.

One thing you can notice with ChatGPT, particularly when you're playing around with jailbreaking it, is that the start and end of its responses seem to be much more tame than the middle. It feels almost like there's a force steering its latent vector towards a specific region of political correctness, which then lets up, and picks up again in the final paragraph with its uncontrollable urge to either provide a summary of what it just said, or remind you that the thing it just taught you to do (hacking, robbing a bank, etc) is illegal and shouldn't be attempted.

It's certainly not a "text replacement" type of thing or even detecting sentiment and taking action. If it were less pronounced it might even not be noticeable, but it very much looks like it integrates with the model and acts within its weights.

One can imagine a dystopic world in which society relies on a big model that's inarguably smarter than humans, and the model is being subtly influenced in such a way, perhaps only when interacting with users identified as susceptible, sympathetic, or relevant to a certain goal. And what's concerning is how that doesn't really seem too far off at all (OpenAI's political biases currently infused into GPT as a real current example notwithstanding).

Re: And yet It Understands

#97

Post by entropyneur »

I think the question of whether AI has "true understanding" of things is misguided. Having a "true understanding" is nothing but a subjective experience. There are two actual important questions: 1) whether AI is capable of having (any) subjective experience at all and 2) whether AI can outperform human intelligence in every area. You are in a deep denial if in 2023 you have any doubts about 2). I'm yet to hear a compelling argument as to why a positive answer to 2) might imply a positive answer to 1). However it's appalling how little attention is being given to 1) on it's own merit.

Re: And yet It Understands

#98

Post by smitty1e »

doctor_eval wrote: post #63
smitty1e wrote: post #24

> But nobody knows how GPT works. They know how it was trained, because the training scheme was designed by humans, but the algorithm that is executed during inference was not intelligently designed but evolved, and it is implicit in the structure of the network, and interpretability has yet to mature to the point where we can draw a symbolic, abstract, human-readable program out of a sea of weights. I object. ChatGP…

So what if it's electrical signals. It's not really 1s and 0s is it, they are grouped together into floating point numbers, right? And at some point there are enough ones and zeroes that you can effectively simulate just about any analog state. So why should we assume it's not possible to simulate the states of neurons? And why do we assume that the brain's structure is the only way to produce intelligence? Surely -…

Well, if we're to believe that thr Philosopher's Stone has arrived, why should it not have oracular power?

I am regularly assured by the Climatistas that the apocalypse is at hand. What saith ChatGPT?

Re: And yet It Understands

#99

Post by raincole »

bmacho wrote: post #93
raincole wrote: post #73

Earlier quoted context omitted.

I tried it several (>10) times and I couldn't get the input suggestions. https://i.imgur.com/SaVjSzT.png Possibility 1: It's an extremely rare case and I need to try more Possibility 2: MS patched it quickly when they saw this article Possibility 3: The author lied for page views Choose your poison.

I think the very first appearance of this image is 18 February [0], Microsoft could have patched it since. There are several other examples in the comments. [0] : https://www.reddit.com/r/bing/comments/1150po5/sydney_tries_...

I see. If it was a month ago then MS would have almost definitely patched it.
Post reply on HN