61–70 of 83 posts
Earlier quoted context omitted.
Perhaps but I’m pretty confident a ChatGPT is not “it” this time.
Sortie's Paradox.
Earlier quoted context omitted.
Sortie's Paradox.
My confidence comes from watching GothamChess play it at… chess. The problem with ChatGPT isn’t that it’s bad at chess or doesn’t understand the rules of the game. It obviously has no concept of what it’s doing. It can’t even keep track of which pieces are still on the board: a trivial task for even a small child.
Push out a plugin that sets up a virtual board GPT-4 can read after each move and see if its any better.
The Forer Effect can't make chatGPT generate syntactically correct and functional code snippets that I use every day at work.
Let me make this very clear: Generating syntactically correct code in various languages that also looks plausible is no small feat. It is, in fact, extremely impressive and will certainly have an impact on SE. But. Every single test I ran lead to functionally wrong designs from smallish memory errors in C (that hilariously ChatGPT was able to correct ND explain when pointed to) over misplaced/hallucinated methods in…
Earlier quoted context omitted.
Let me make this very clear: Generating syntactically correct code in various languages that also looks plausible is no small feat. It is, in fact, extremely impressive and will certainly have an impact on SE. But. Every single test I ran lead to functionally wrong designs from smallish memory errors in C (that hilariously ChatGPT was able to correct ND explain when pointed to) over misplaced/hallucinated methods in…
I have the same experience. I think the transformer-based models are impressive, but whenever I try to actually use them for something, e.g. having ChatGPT write something or using GitHub Copilot, the results I get are terrible. I was legitimately afraid for my job when GPT-4 was released. After using it, however, I sleep easy now.
Just wait until GPT-5, though. It will blow your mind... \s
Earlier quoted context omitted.
That’s not really an issue when we’re talking about AI. I said in a separate comment on the topic: intelligence is such a complicated thing that we seem to only be able to define it by pointing to things and saying “that’s not it.” If we didn’t move the goalposts, we’d have declared Stockfish to be full AI, despite it only being a chess-playing program, long ago.
We agreed on a definition of artificial intelligence--the Turing test--for 50 years. The goalpost was clearly established, widely agreed upon, and promptly abandoned when chatbots blew past it. I'm convinced that when the dust has settled and historians look back to decide on THE point in time at which we achieved AI or even AGI, that time will not be in the future, but in the past.
If historians are looking back to decide the point, by definition doesn't it have to be in the past? Historians looking back to the future doesn't make sense...
Earlier quoted context omitted.
Let me make this very clear: Generating syntactically correct code in various languages that also looks plausible is no small feat. It is, in fact, extremely impressive and will certainly have an impact on SE. But. Every single test I ran lead to functionally wrong designs from smallish memory errors in C (that hilariously ChatGPT was able to correct ND explain when pointed to) over misplaced/hallucinated methods in…
I have the same experience. I think the transformer-based models are impressive, but whenever I try to actually use them for something, e.g. having ChatGPT write something or using GitHub Copilot, the results I get are terrible. I was legitimately afraid for my job when GPT-4 was released. After using it, however, I sleep easy now.
I was using it to try and help me create Arma3 scenarios, which often involves using an ugly little DSL called SQF.
The last thing I asked it about was how to play a custom audio file sitting on my disk. It gave me a simple one-liner, that didn’t do anything. I asked it several questions to clarify, but it seemed confident that all I needed was this one liner to play an audio file sitting anywhere on my disk.
Eventually I said fuck it, and decided to google it myself, and it turns out it’s much more involved than just a single function call with a path as an argument, as suggested by ChatGPT. In fact you have to create an entirely separate config file with classes to represent the audio, and then you can reference that class to play the audio, not just a simple string path.
I’ve had it fail with a lot of other things, mostly trying to make computer controlled units perform deterministic, scripted actions, but it’s hard to tell if GPT-4 or the game’s engine was at fault here, because in my experience making Arma’s AI units do anything reliably is near impossible, no matter how much scripting you do.
Earlier quoted context omitted.
My confidence comes from watching GothamChess play it at… chess. The problem with ChatGPT isn’t that it’s bad at chess or doesn’t understand the rules of the game. It obviously has no concept of what it’s doing. It can’t even keep track of which pieces are still on the board: a trivial task for even a small child.
Small children track in their 'minds eye' the entire layout of a chess board and where each piece is? Tokenization starts screwing stuff up if you're start printing out a board after each move. Push out a plugin that sets up a virtual board GPT-4 can read after each move and see if its any better.
My point is that the hyperbole about “we’ve cracked AI but they changed the goalposts” is self evidently not true. You just proved it right there: I need to add more plugins because ChatGPT does not understand what it’s doing.
It’s a potentially useful tool but that doesn’t make it intelligent.
The Forer Effect can't make chatGPT generate syntactically correct and functional code snippets that I use every day at work.
The Forer Effect can't make chatGPT generate syntactically correct and functional code snippets that I use every day at work.
Let me make this very clear: Generating syntactically correct code in various languages that also looks plausible is no small feat. It is, in fact, extremely impressive and will certainly have an impact on SE. But. Every single test I ran lead to functionally wrong designs from smallish memory errors in C (that hilariously ChatGPT was able to correct ND explain when pointed to) over misplaced/hallucinated methods in…