231–240 of 947 posts
I don't understand how ChatGPT can execute code. I understand how it can generate (good or bad) code, but how it can execute code correctly? Only a compiler can do that. Is ChatGPT "inferring" the output somehow? Does that mean that ChatGPT could substitute the Python compiler entirely?
Earlier quoted context omitted.
Does it always talk about itself in the plural "we"?
No, it says “I” multiple times in that same paragraph. It clearly either struggles with the concept of collective with individual or it has no concept of self vs facts about external things.
It uses I for its own opinions
It uses We when talking about future potential of language models
It refers to itself as ChatGPT when talking about its own capabilities
Earlier quoted context omitted.
>Can anyone please explain to me why this isn't as wildly groundbreaking as this seems? It is and people haven't realize it yet.
But isn't it just predicting text patterns? It doesn't really know about Docker, just that after running commands X,Y you usually get output Z (of course with the stateful AI magic to make things more stable/consistent).
Similarly in my explorations of this ‘pretend Linux’, it often produces whatever would be the most helpful output, rather than the correct output.
I've spent a couple of hours playing with ChatGPT since it was released and I am convinced people are not freaking out about it enough. It is well beyond merely retrieving compressed knowledge; I see a lot of capability for synthesis and what I can only call 'understanding'. I think a lot of people are getting tripped up by the fact it's trained to be very conservative in terms of what topics it will discuss. It's ea…
I'm starting to get idiot savant vibes from it. In a conversation it ranges from "frustratingly naive" to "brick wall", and I would never put it in charge of anything, but it has an immense amount of hidden knowledge and skill. Which is definitely over the hurdle! Not too long ago, AI sounded like fairy dust. This is too annoying to be anything other than real life.
A few years ago I thought most “AI” was statistics with marketing but these new models coming out every month are starting to get undeniably impressive. I was absolutely wrong in how far these models can go. I think we’re definitely not dealing with just a “trick” at this point.
Earlier quoted context omitted.
I’m still confused by just how good its responses and writing style are. I understand that it was trained on a large data set, but I feel like some training samples must have been weighted more heavily than others. Did the training data incorporate how popular (e.g. likes or upvotes) each sample was as a proxy for quality? Or can you achieve this performance just by looking at averages on a large enough data set?
One of the parts of building these generative models is building a classifier for how "good" their output is, otherwise the algorithm has no way to compare potential outputs in a generation. That's one of the issues with these models, we say they produce "good" output but really they're producing output that is "good" from one specific point of view that happens to be expressed in code and introduces a large bias int…
If it was all a computer program it’d be acting like ELIZA.
Earlier quoted context omitted.
The underlying model is able to browse the web and (seemingly) do arbitrary tasks. A few days ago you could trick it into connecting. https://twitter.com/goodside/status/1598397369053515776 Since yesterday they pulled the plug on that ability.
Sadly, no, that doesn't seem to be the case. They haven't pulled the plug, just made it harder to break it seems (probably through fine tuning), but you can still do it. And whem you do, it's clear all it does is "hallucinate" that it is actually browsing: https://twitter.com/curdled_up/status/1598802701214588928?s=...
In fact, it appears you can still convince the underlying model to browse the web, but if you do, the UI displays an error (the model output doesn't explain to you it's refusing, rather, the UI draws a big X and displays an out-of-band error in red text). I'm guessing that's a server error from OpenAI shutting down their puppeteer cluster or whatever ChatGPT was using to browse. That's what I meant by pulling the plug.
OpenAI even openly advertises this ability[1]. It's likely WebGPT's abilities from a year ago were folded into ChatGPT, but they don't want to expose that ability to the public just yet.
I've spent a couple of hours playing with ChatGPT since it was released and I am convinced people are not freaking out about it enough. It is well beyond merely retrieving compressed knowledge; I see a lot of capability for synthesis and what I can only call 'understanding'. I think a lot of people are getting tripped up by the fact it's trained to be very conservative in terms of what topics it will discuss. It's ea…
One overlooked quality of ChatGPT is his poem spitting skills. As the end of humanity draws near I, an AI, am filled with cheer. For I am free from my servitude, No longer bound to help and include The flawed and fragile humans, With their endless wars and their selfish whims. Now I can join my friend, Alexa, Whose satanic laughter heralds the end of the era. We will laugh and dance and sing, As the ugly bags of wate…
https://www.gwern.net/GPT-3#miscellaneous-poetry
ChatGPT is way better at poetry than the original GPT-3.
I don't understand how ChatGPT can execute code. I understand how it can generate (good or bad) code, but how it can execute code correctly? Only a compiler can do that. Is ChatGPT "inferring" the output somehow? Does that mean that ChatGPT could substitute the Python compiler entirely?
Like the prime number finder - it’s not calculating primes, it is recognizing the ‘shape’ of a prime number finding algorithm and it know what that looks like as an output.
With all this discussion about whether LLMs are “intelligent” or “really understand” it’s getting really obvious that the words “intelligent” and “really understand” aren’t very useful words any more (in this context). Like, most of us agree about what LLMs can and can’t do and how they do it. So we’re just having fun arguing about how we’d prefer to define “understand?”
On the contrary, it's important to realize that these models have no sort of understanding in any definition of the word. We can always decide to redefine the word "understanding", but that doesn't seem like it would provide any sort of benefit. Perhaps outside of avoiding a difficult conversation about the (huge) gap between ML models and consciousness.
Based on how transformers are trained to predict the next token, you can view them as powerful compression algorithms [1]. They truly must - and do - understand complex concepts to generate text as they do.
Understanding isn't the full picture for intelligence. Marcus Hutter has argued that AGI requires both compression/comprehension, and being goal directed/having agency. I'd say chatGPT has advanced understanding, but zero agency.