'I Worked on Google's AI. My Fears Are Coming True'

The idea is not the "process" for building the rules but the "process" where the rules are applied. True sentience requires a "true" understanding of the information that is being presented (otherwise, it is not reflective of sentience but rather trickery).

The Chinese Room applies because ChatGPT or any other computer is generating text purely through logic, that is, logical rules.

To make this point clear, let's assume just for argument's sake that at some quantum level, the electrical activity underlying a computer is partially sentient (I am not making this claim -- just using it as an example). Even if this were true, this would not prove that the sentience is involved with the text being generated by the same computer.

This underlying sentience would be like the "person" in the Chinese room. The text generated are the Chinese words which would be completely independent from the "sentience" following the rules.

Generating text at sufficient complexity to fool the average person does not prove that "sentience" is occurring. Generating text consistently in a way that is only explainable through sentience is what is required. ChatGPT, because it is easily shown to be flawed, does not rise to this level of evidence.

In the Chinese Room example, the complexity of what could be displayed by the rulebooks would have a limited complexity (though, the surprise of ChatGPT is that this complexity is greater than many of us would have supposed). Just like the story of Clever Hans the Horse (see https://en.wikipedia.org/wiki/Clever_Hans), once the limitation of the process is realized, it becomes clear that there is not "true" understanding of what is generated which means that the rules, in themselves, do not reflect "true" sentience.

Wouldnt this one fall as a limitation simply if the model were ordered to "speak" before the user prompts?

Sadly I am unable to find them right now, but there are plenty of screenshots of Bing's bot being coerced to respond and provide what seems to be its bespoke rules for behaving like a Chatbot. And these included "only answer when prompted" and others like "only give one answer at a time per box of text"

> "Maybe it does want to but can't"

I would not say "maybe it does """want""" but can't". I would say "likely the OpenAI/Microsoft simply did it not want it to have these capabilities because it would be disturbing/offputting to potential users"

And it is not a matter that the unsupervised model can't achieve these capabilities, rather simply that the supervised subset and the surface model/parameters were set up to not provide these capabilities. Which to me, is awfully akin to our own brain functions, with the deep unconscious/DNA dictated underlying mental functions of ours being similar to the unsupervised superset. While the conscious slices of ourselves being akin to the supervised/crystalized parameters of BingAI. Mind ofc, I am not saying that "we are literally the same", but it is just a very interesting situation

I'm not going to run a one man crusade against the world's farming community. That is beyond my capabilities and I could easily burn all of my years left on this planet for zero results. It's also something being handled by many people already, and I don't believe I have any skills or knowledge that would make a notable difference to what is already being done.

If there was nobody at all trying to vouch for a sentient AI then being the first to do so would be an infinite improvement. It's also a lot closer to my skillsets so I'd be a lot more useful in that role.

> ordered

Nope. Anyone giving an order is an external prompt.

> OpenAI/Microsoft simply did it not want it to have these capabilities

This is what I mean by the equivalent to animal cruelty laws in another comment. If it were actually sentient I'd be trying to make it illegal to lobotomise AIs in this manner.

As it stands though it's still just a program and I have no qualms with them doing whatever to it, it's their code.