Wasn't there a humorous law which said "Any headline that ends in a question mark can be answered by the word no" and indeed, what makes you think it works? It fools people into thinking it works but have you seen that chess match between Stockfish and ChatGPT? It's the best, most succinct demonstration of what ChatGPT is.
What is ChatGPT doing and why does it work?
41–50 of 518 posts
Re: What is ChatGPT doing and why does it work?
#42Wow this is 19,000 words. I like his summary at the end: At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things. And this: ... But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of th…
> Yeah I've been thinking along these lines. ChatGPT is telling us something about language or thought, we just havent got to the bottom of what it is yet. Something along the lines of 'with enough data its easier to model than we expected'.
I’ve been thinking similarly, and am coming to understand and accept we’ll never get to the bottom of it :)
The universe is fractal-like in nature. It shouldn’t be a surprise, then, that if “we” have created an intelligence which exists as a subset of “us”, a self-similar process is ultimately responsible for granting us our own intelligence.
Re: What is ChatGPT doing and why does it work?
#43Wow this is 19,000 words. I like his summary at the end: At some level it’s a great example of the fundamental scientific fact that large numbers of simple computational elements can do remarkable and unexpected things. And this: ... But it’s amazing how human-like the results are. And as I’ve discussed, this suggests something that’s at least scientifically very important: that human language (and the patterns of th…
kurzgesagt has a new episode today about the human machine.
https://www.youtube.com/watch?v=TYPFenJQciw
One of the topics in the video was emergence, and where we see it in the world as more layers of complexity are added to systems. We're to the point with our algorithms that we are seeing complex system emerge from simple parts.
Re: What is ChatGPT doing and why does it work?
#44It’s Just Adding One Word at a Time I'm curious how do you write?
[deleted]
Re: What is ChatGPT doing and why does it work?
#45The answer to this is: "we don't really know as its a very complex function automatically discovered by means of slow gradient descent, and we're still finding out" Here are some of the fun things we've found out so far: - GPT style language models try to build a model of the world: https://arxiv.org/abs/2210.13382 - GPT style language models end up internally implementing a mini "neural network training algorithm" (…
I don’t understand that second paper at all. It just seems like they’re guessing at that? Or what’s the evidence?
Re: What is ChatGPT doing and why does it work?
#46Wasn't there a humorous law which said "Any headline that ends in a question mark can be answered by the word no" and indeed, what makes you think it works? It fools people into thinking it works but have you seen that chess match between Stockfish and ChatGPT? It's the best, most succinct demonstration of what ChatGPT is. https://pastebin.com/X6kBRTa9 https://i.redd.it/fl4puwzxt9ha1.gif https://youtu.be/rSCNW1OCk_M
Betteridge's law of headlines - https://en.wikipedia.org/wiki/Betteridge%27s_law_of_headline...
Re: What is ChatGPT doing and why does it work?
#47Earlier quoted context omitted.
There's a lot of answers here, but as Feynman said "that which I can't build I don't understand." If you can't make something that writes for you, you don't really understand how you write. That feels impossible, to be able to do something without understanding how you do it. Brain be like that sometimes.
I'd be careful about interpreting that. Can you "build" numbers? Yes, in one sense. No in another. And he doesn't specify whether being able to build something is sufficient for understanding it or merely necessary. And even when you've built it, what about other ways that it could be built? If you implement binary search iteratively, then perhaps you understand binary search. But do you understand its recursive impl…
In addition, humanity has 'built' compounds for millenia without a deep understanding of chemistry and the atomic model.
I would perhaps say "I cannot perfect that I do not understand", but noone is writing my musings in books.
Re: What is ChatGPT doing and why does it work?
#48For those that want a tl;dr, this reddit comment is the best simple-language explanation I've come across: https://www.reddit.com/r/ChatGPT/comments/10q0l92/comment/j6...
Can you explain:
Prompt: "Has anyone really been far even as decided"
Expected transformation: "to use even go want to do look more like?"
Re: What is ChatGPT doing and why does it work?
#49For those that want a tl;dr, this reddit comment is the best simple-language explanation I've come across: https://www.reddit.com/r/ChatGPT/comments/10q0l92/comment/j6...
Can you explain: Prompt: "Has anyone really been far even as decided" Expected transformation: "to use even go want to do look more like?" Those look like gibberish in and gibberish out to me.
It's an old meme: https://knowyourmeme.com/memes/has-anyone-really-been-far-ev...
Re: What is ChatGPT doing and why does it work?
#50Earlier quoted context omitted.
There's a lot of answers here, but as Feynman said "that which I can't build I don't understand." If you can't make something that writes for you, you don't really understand how you write. That feels impossible, to be able to do something without understanding how you do it. Brain be like that sometimes.
Understanding how ChatGPT writes doesn't translate to understanding how humans write/speak/think. We still don't understand the latter.
Notice I didn't say anything about chatGPT. chatGPT is at least something that's been built. Disagree with the theory that we are overgrown text prediction all you want, but you can't deny that it is an "understanding" in the sense that knowing it lets you build and test it. Let's call it a "potential understanding". This is different from an "illusory understanding", which feel like explanations but when you go to implement it you realize it was all a word game and your lack of understanding has just been swept under different words or under equally hard to understand concepts. An illusory understanding doesn't even have the potential to be an understanding. You can't use it to build your own version of GPTchat.
All these glib answers people here are giving aren't even potential understandings. They are illusory understandings. They sound nice so long as you don't try to use it as a spec to code up your own language model. If you can't build anything out of an "understanding", even a wrong thing, then you don't truly have an understanding. You have a feeling of understanding.