Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building a Virtual Machine Inside ChatGPT

engraved.blog

Post reply on HN
Print view View original

261–270 of 947 posts

Re: Building a Virtual Machine Inside ChatGPT

#261

Post by coffeebeqn »

carl_dr wrote: post #198
mertd wrote: post #97

Earlier quoted context omitted.

I see trust as the big issue. Yes, it gives very reasonable answers. But how do I know it is not BSing, defrauding, or tricking me? You can ask for it to reveal its sources, but then you're back to Google style knowledge gathering of checking websites and judging credibility.

> How many ways can you permute the letters in the word “safer”? Starts out just wrong, then explains the working and correct answer, but just completely loses it by the end. That calculation!

It seems to not calculate math formulas. Probably they turned that off because people would throw absurdly expensive operations at it

Re: Building a Virtual Machine Inside ChatGPT

#262

Post by csande17 »

dragonwriter wrote: post #212
csande17 wrote: post #173

Earlier quoted context omitted.

ChatGPT got a 1020 on SAT Practice Test #10 ( https://satsuite.collegeboard.org/media/pdf/sat-practice-tes... ), a sample exam whose questions have been extensively discussed online since at least 2019 (eg https://youtube.com/watch?v=18oahUy6cGA ). The exact text of the questions, with the corresponding correct answers, almost certainly appeared in ChatGPT's training set.

Prompt: Would a chatbot scoring 1020 on an SAT practice test whose questions and answers were part of its training data be evidence of general understanding? ChatGPT Response: A chatbot scoring 1020 on an SAT practice test whose questions and answers were part of its training data would not necessarily be evidence of general understanding. Scoring well on a standardized test like the SAT requires a specific set of sk…

I do wonder how output like this would fare on the essay section. All the spelling and grammar is correct, and ChatGPT can presumably generate as many paragraphs of it as you want. (And I think that's what SAT graders mostly look for in practice.) But it also repeats some generic sentence about "a standardized test like the SAT requiring specific skills and knowledge" twice with superficial avoiding-a-plagiarism-checker alterations, takes until the second paragraph for it to figure out what point it's making, and uses the wrong conjunction when it's making that point.

Re: Building a Virtual Machine Inside ChatGPT

#263

Post by coffeebeqn »

Xelynega wrote: post #183
CamperBob2 wrote: post #40

Earlier quoted context omitted.

Agreed, this will force a rethink of several aspects of what we currently call intelligence. The dialog in my head tends to play out like this: "This thing is scary. It mimics human thought, including human creativity, amazingly well." "Nah, don't worry, it's just a black-box language model. A dumb machine that emits one word at a time with no concept of meaning. The underlying logic is nothing but a probabilty distr…

If a probabilistic process is able to 'pass a Turing test', the only thing that really says is that the Turing test wasn't accomplishing the stated goal so we need to come up with a better test. Sidenote, but the idea that it's "mimicking human thought" is wrong in the first place and an anthropomorphism that comes from all the marketing calling it "ai" instead of "generative language models". I don't know how any hu…

The whole point of the turing test is that if you can’t tell the difference then something interesting has happened. You can say it’s “just X” or “just Y” but I think you’re being overly dismissive of the accomplishment here. If we’re at a point where we say, sure it passes the Turing test but moves goalposts - then that’s pretty exciting!

Re: Building a Virtual Machine Inside ChatGPT

#264

Post by dragonwriter »

nl wrote: post #109
sandgiant wrote: post #103

Earlier quoted context omitted.

Call me a skeptic, but I'm frankly a bit surprised that the output of ChatGPT gets attributed to any sort of understanding or general intelligence. Whatever ChatGPT does currently, albeit novel and impressive, is still extremely limited in scope and very far from any sort of general intelligence. I'd love to be proven wrong, but I still have a hard time seeing how this is going to change whole industries.

"general intelligence" means so many things to different people. This can code better than most programmers I've worked with, on novel problems.

ChatGPT seems to disagree with this assessment.

Prompt: Can you code better than most programmers?

Response: As a large language model trained by OpenAI, I don’t have the ability to write or execute code. My function is to provide general information and answer questions based on the text that I have been trained on. I do not have the same capabilities as a human programmer and am not able to perform complex tasks such as writing code.

Re: Building a Virtual Machine Inside ChatGPT

#265

Post by visarga »

jameshart wrote: post #171
davidktr wrote: post #65

Earlier quoted context omitted.

Yes, but it cannot really have conversations it has not read about on the web. There is no real abstraction. It excells at outputting symbols in the correct order, given we have an idea what the pattern should look like. Otherwise it's stuck in a loop.

Go back and reread the original post here. Examine carefully the output it produces for these curl commands. Explain where on the web it would have found those during its training.

It learns concepts and how to recombine them. That means it interpolates in a very sparse space. Most concept combinations have not been thought yet, so almost every time it samples it looks original.

Re: Building a Virtual Machine Inside ChatGPT

#266

Post by codeulike »

sandgiant wrote: post #131
6gvONxR4sf7o wrote: post #112

With all this discussion about whether LLMs are “intelligent” or “really understand” it’s getting really obvious that the words “intelligent” and “really understand” aren’t very useful words any more (in this context). Like, most of us agree about what LLMs can and can’t do and how they do it. So we’re just having fun arguing about how we’d prefer to define “understand?”

On the contrary, it's important to realize that these models have no sort of understanding in any definition of the word. We can always decide to redefine the word "understanding", but that doesn't seem like it would provide any sort of benefit. Perhaps outside of avoiding a difficult conversation about the (huge) gap between ML models and consciousness.

It's not that we don't understand that this is just a language model.

We do understand that.

The fact that a language model can produce convincing output hints that it is not that different from us.

Imagine something in 5 or 10 years time with 1000 times the power. What will it be like?

I know you think our awareness makes us fundamentally different. But what if it doesn't? What if we're just a machine that likes to anthropomorphize things reflected back on itself?

Ultimately, we won't have to know the answer.

In 10 years AIs like this will be in computer games. Some people will find the results of that disturbing. Is highly accurately similated suffering different to real suffering? The philosophical answer won't matter, the practical answer will be that AI use in games will become regulated. And then you have the first glimmer of AIs getting rights.

Re: Building a Virtual Machine Inside ChatGPT

#267

Post by coffeebeqn »

davidktr wrote: post #65
CamperBob2 wrote: post #40

Earlier quoted context omitted.

Agreed, this will force a rethink of several aspects of what we currently call intelligence. The dialog in my head tends to play out like this: "This thing is scary. It mimics human thought, including human creativity, amazingly well." "Nah, don't worry, it's just a black-box language model. A dumb machine that emits one word at a time with no concept of meaning. The underlying logic is nothing but a probabilty distr…

Yes, but it cannot really have conversations it has not read about on the web. There is no real abstraction. It excells at outputting symbols in the correct order, given we have an idea what the pattern should look like. Otherwise it's stuck in a loop.

This is not what’s happening. It’s not just regurgitating text it has read.

There are whole classes of problems to solve still but it’s safe to say this will be good enough to solve some of the first steps

Re: Building a Virtual Machine Inside ChatGPT

#269

Post by ves »

fergal_reid wrote: post #15

I've spent a couple of hours playing with ChatGPT since it was released and I am convinced people are not freaking out about it enough. It is well beyond merely retrieving compressed knowledge; I see a lot of capability for synthesis and what I can only call 'understanding'. I think a lot of people are getting tripped up by the fact it's trained to be very conservative in terms of what topics it will discuss. It's ea…

It sounds good, except that there’s no results for “screechings” or “only cure” in the books, and Ron doesn’t encounter Voldemort in Goblet, only Harry and Cedric do.

Again, it has no semantic comprehension, it doesn’t know that certain strings of words are valid in certain contexts because they refer to actual events, objects, or concepts, it’s just putting letters together.

Probably some of its training material included Harry Potter fan fiction or other non-canon stuff, and that’s why it said what it said.

Re: Building a Virtual Machine Inside ChatGPT

#270

Post by visarga »

EGreg wrote: post #74
davidktr wrote: post #65

Earlier quoted context omitted.

Yes, but it cannot really have conversations it has not read about on the web. There is no real abstraction. It excells at outputting symbols in the correct order, given we have an idea what the pattern should look like. Otherwise it's stuck in a loop.

Step 1: Have humans commit all kinds of knowledge to the Web for 20 years Step 2: Analyze it. Step 3: Make a model that regurgitates it and comes up with new variations. Humans are surprised it's so "human-like" in its responses, and anthropomorphosize it to believe it "understands" what it's writing, when it in fact is remixing bits of pieces of that a billion other humans wrote over the years. And yet ... perhaps t…

There have been a few more steps, and lots of virtual ink on ML papers.
Post reply on HN