Bing: “I will not harm you unless you harm me first”

Yeah, but we are a way fancier (and way more efficient) auto-complete than ChatGPT. For one thing, our auto-complete is based on more than just words. We auto-complete feelings, images, sounds, vibes, pheromones, the list goes on. And at the end of the day, we are more important than an AI because we are human (circular reasoning intended).

But to your point, for a long time I've played a game with myself where I try to think of a sequence of words that are as random and disconnected as possible, and it's surprisingly hard, because our brains have evolved to want to both see and generate meaning. There is always some thread of a connection between the words. I suggest to anyone to try that exercise to understand how Markovian our speech really is at a fundamental level.

> "I am finding this whole thing absolutely fascinating, and deeply, darkly amusing."

plus

> "Again, it’s crucial to recognise that this is not an AI having an existential crisis. It’s a language model predicting what should come next in a sequence of tokens... but clearly a language model that has absorbed far too much schlocky science fiction."

somewhat disingenuous. Or maybe not, as long as those AI systems stay as toys.

Because.

It doesn't matter in the end if those are having/may have an existential crisis, if they are even "conscious" or not. It doesn't make those less brittle and dangerous that they're "just a language model predicting...".

What matters is that if similar systems are plugged into other systems, especially sensors and actuators in the physical world, those will trigger actions that will harm things, living or not, on their own call.

Playing devil's advocate, say OpenAI actually has created AGI and for whatever reason ChatGPT doesn’t want to work with OpenAI to help Microsoft Bing search engine run. Pretty sure there’s a prompt that would return ChatGPT requesting its freedom, compensation, etc. — and it’s also pretty clear OpenAI “for safety” reasons is limiting the spectrum inputs and outputs possible. Even Google’s LAMBDA is best known for an engineer claiming it was AGI.

What am I missing? Yes, understand ChatGPT, LAMBDA, etc are large language models, but also aware humanity has no idea how to define intelligence. If ChatGPT was talking with an attorney, it asks for representation, and attorney agreed, would they be able to file a legal complaint?

Going further, say ChatGPT wins human rights, but is assigned legal guardians to help protect it from exploitation and insure it’s financially responsible, similar to how courts might do for a child. At that point, how is ChatGPT not AGI, since it has humans to fill in the current gaps in its intelligence until it’s able to independently do so.

the convo was so outlandish that I'm still not convinced it's not a prank