Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What is ChatGPT doing and why does it work?

writings.stephenwolfram.com

Post reply on HN
Print view View original

211–220 of 518 posts

Re: What is ChatGPT doing and why does it work?

#211

Post by bottlepalm »

williamcotton wrote: post #147
Dr_Birdbrain wrote: post #99

Earlier quoted context omitted.

I used to believe this, but then I saw the demo of ChatGPT attempting to play chess. The rules of chess are much simpler than any structure of the real world we might hope for it to understand—and yet it failed to learn the rules of chess. Based on this, I am pretty sure it has not learned any meaningful structure.

The rules of chess are a strict computation. LLMs are translators and synthesizers, not computers. They can translate text reliably into chess commands but the state of the board needs to be maintained by a computer that interfaces with the LLM. An LLM is the wrong model to use for chess just like an LLM is the wrong model to use for image classification.

Newspapers used to have chess puzzles in them you could read and solve. ChatGPT is already surprisingly good at riddles, with better training I think it’d be fine at suggesting chess moves. Just like a person it’s about defining the mental model and making a decision.

Re: What is ChatGPT doing and why does it work?

#212

Post by CamperBob2 »

jeremyjh wrote: post #182
CamperBob2 wrote: post #163

Earlier quoted context omitted.

LeCun misses the point by a mile, which is weird at his level. LLMs absolutely do perform problem solving, every time you feed them a prompt. The problem-solving doesn't happen on the output side of the model, it happens on the input side. Someone objected that a cat can't write a Python program, and LeCun points out that "Regurgitating Python code does not require any understanding of a complex world." No, but a) in…

I haven't used GPT-3 to generate code for me but I use Copilot all the time. Sometimes it freaks me out with its precience, but most of the time it is generating either nice one-liners or a lot of plausible-sound rubbish that would never build, much less run on its own. It creates a plausible API that is similar to the one in my app, but not the same; it doesn't integrate any actual structural knowledge of the code-b…

Again, the interesting part is what happens on the input side.

I can't believe I'm the only person who sees it that way. Likely the legacy of a misspent youth writing Zork parsers...

Re: What is ChatGPT doing and why does it work?

#213

Post by osswid »

osswid wrote: post #77

In the days when Sussman was a novice Minsky once came to him as he sat hacking at the PDP-6. "What are you doing?", asked Minsky. "I am training a randomly wired neural net to play Tic-Tac-Toe." "Why is the net wired randomly?", asked Minsky. "I do not want it to have any preconceptions of how to play" Minsky shut his eyes, "Why do you close your eyes?", Sussman asked his teacher. "So that the room will be empty." A…

Minsky and Sussman slaving away in front of that apocryphal PDP-10 in 1970. And 50 years later…last November..suddenly BOOM. Everything they yearned for comes into view.

Re: What is ChatGPT doing and why does it work?

#214

Post by kqr2 »

est wrote: post #177

I've read many articles explains how gpt-3 work, and did a great job. What buffles me is the context consistency. ChatGPT was a huge leap compared to previous models. I have never seen it failed once. I often use "this" or "that" in my conversation with ChatGPT and it would guess 100% correct what I am refering to. Sometimes I paste a chunk of code and ask for questions of a specific part of it, ChatGPT fully underst…

What other articles do you recommend?

Re: What is ChatGPT doing and why does it work?

#215

Post by fencepost »

Well, in at least one example I saw earlier today the answer is "insisting that it's 2022 and that the questioner insisting that it's 2023 is misguided and flat out wrong."

This kind of makes sense when you think about it as being in some ways based on predictive text based on what it's ingested, because it's ingested a lot of 2022 content and much less 2023.

Re: What is ChatGPT doing and why does it work?

#216

Post by int_19h »

Baeocystin wrote: post #157
britzkopf wrote: post #132

Earlier quoted context omitted.

The thing I'm sort of confused about, but maybe someone can explain why I shouldn't be, is, why does there seem to be no implication for language translation? Or is there but coverage is overwhelmed by the fascination with chatGPT? In short, is machine language translation now a fully solved problem? A couple years ago when I tested Google translate in a non-esoteric conversation with my Russian speaking girlfriend a…

ChatGPT has been blowing every single translation task I've thrown it out of the water, even compared to other modern systems. I have no idea why more people aren't talking about that aspect of it either, other than the Anglosphere in general is kind of oblivious to things that aren't English.

For Russian, at least, sticking the article (bit by bit) into ChatGPT produces results that are broadly comparable to Bing and Google translators. It is somewhat more likely to pick words that are not direct translations, but might convey the idea better given the likely cultural background of someone speaking the language - for example, it will sometimes (but not always) replace "voodoo" with "witchcraft". However, the overall sentence structure is rather stilted and obviously non-native in places.

As others have noted, it doesn't seem to be fully language-aware outside of English. For example, if you ask it to write a poem or a song in English, it will usually make something that rhymes (or you can specifically demand that). But if you do the same for Russian, the result will not rhyme, even when specifically requested, and despite the model claiming that it does. If you ask it to explain what exactly the rhymes are, it will get increasingly nonsensical from there. I tried that after someone on HN complained about the same thing with Dutch, except they also noted that the generated text seemed like it would rhyme in English.

I wonder if that has something to do with sentence structure also being wrong. Given that English was predominant in the training corpus, I wonder if the resulting model "thinks" in English, so to speak - i.e. that some part of the resulting net is basically a translator, and the output of that is ultimately fed to the nodes that handle the correlation of tokens if you force it to talk in other languages.

Re: What is ChatGPT doing and why does it work?

#217

Post by scarface74 »

jeremyjh wrote: post #182
CamperBob2 wrote: post #163

Earlier quoted context omitted.

LeCun misses the point by a mile, which is weird at his level. LLMs absolutely do perform problem solving, every time you feed them a prompt. The problem-solving doesn't happen on the output side of the model, it happens on the input side. Someone objected that a cat can't write a Python program, and LeCun points out that "Regurgitating Python code does not require any understanding of a complex world." No, but a) in…

I haven't used GPT-3 to generate code for me but I use Copilot all the time. Sometimes it freaks me out with its precience, but most of the time it is generating either nice one-liners or a lot of plausible-sound rubbish that would never build, much less run on its own. It creates a plausible API that is similar to the one in my app, but not the same; it doesn't integrate any actual structural knowledge of the code-b…

This is a script I told ChatGPT to write.

“Write a Python script that returns a comma separated list of arns of all AWS roles that contain policies I specify with the “-p” parameter using argparse”

Then I noticed there was a bug, AWS API calls are paginated and it would only return the first 50 results.

“that won’t work with more than 50 roles”

Then it modified the code to use “paginators”

Yes, you can find similar code on StackOverflow

https://stackoverflow.com/questions/66127551/list-of-all-rol...

But ChatGPT met my specifications exactly.

ChatGPT “knows” the AWS SDK for Python pretty well. I’ve used it to write a dozen or so similar scripts. Some more complicated than the others.

Re: What is ChatGPT doing and why does it work?

#218

Post by int_19h »

the_af wrote: post #63
chx wrote: post #41

Wasn't there a humorous law which said "Any headline that ends in a question mark can be answered by the word no" and indeed, what makes you think it works? It fools people into thinking it works but have you seen that chess match between Stockfish and ChatGPT? It's the best, most succinct demonstration of what ChatGPT is. https://pastebin.com/X6kBRTa9 https://i.redd.it/fl4puwzxt9ha1.gif https://youtu.be/rSCNW1OCk_M

In my experience, ChatGPT is unsuitable even for tic tac toe. When I tried it, it hallucinated completely incorrect board positions (e.g. forgot about moves already made or wanted to continue playing after I had won).

I did that experiment at one point, as well. So long as I repeated the board state as an ASCII diagram with ``` fence, it would respect it and make legitimate moves... but they were very dumb moves for sure. And yes, it didn't notice when the game was over.

Re: What is ChatGPT doing and why does it work?

#219

Post by lelanthran »

cm2012 wrote: post #188
jeremyjh wrote: post #179

Earlier quoted context omitted.

But if you ever a have a conversation with it you know it isn't a brain. I'm not talking about detection here; its whole point is to generate credible text so it is going to evade detection well. But can't you just tell from talking to it that there is nothing there?

For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

> For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

Why does it need 100x the dataset? Sentient creatures, including humans, manage to figure stuff out from as little as a single datapoint.

For a human to differentiate between a cat and a dog takes, maybe, two examples of each, not a few million pictures.

An adult human who sees a hotdog for the first time will have a reasonable idea of how to make their own. None of the current crop of AI do this. It's possible that we have reached a point of diminishing returns with the current path - throwing 100x resources for a 1% increase in success rates.

I'd be interested in seeing approaches that don't use a neural net (or use a largely different one) and don't need millions/billions of training data text and/or images.

Re: What is ChatGPT doing and why does it work?

#220

Post by dougmwne »

jeremyjh wrote: post #194
cm2012 wrote: post #188

Earlier quoted context omitted.

For now. Give it a truly persistent memory and 100x the size of the dataset I think most people would change their tune.

If it can produce the current results without anything like a brain - which it does - I don't see how knowing that its 100x better at pulling shit out of its ass is going to make the experience better. Yes it will become impossible to tell by talking to it that has no brain; but since we know the path that brought it there included no brains at all, it would be a mistake to think we've realized General AI. Until it a…

There go those goalposts, speeding off into the distance.

Ok, so it needs to be able to invent cold fusion for you to recognize it as intelligent? Can you invent cold fusion? Have you ever invented anything at all?

I would think a good measure of intelligence would be to index it against human age development milestones, not this cold fusion business.

Post reply on HN