Unfortunately, the hype about LLMs has generated breathless ruminations on AGI and consciousness. Dig a little deeper and there doesn't seem to be much there, as one may find that these terms are not adequately defined. Here we see this attempt: > First, a disclaimer. Consciousness is a notoriously slippery term, if nonetheless possessed of a certain common-sense quality. In some ways, being conscious just means bein…
I think your argument may be a little too strong. Would you refrain from discussing, say, happiness, sadness, etc. in your daily life without a formal definition of what happiness is exactly at a structural level? And I say that as someone interested in formalizing ethics, and formalizing all of those notions. The point is, to even (ideally, formally) define them we need intuition and intuitive discussions about what…
Although humans discuss emotions a great deal, there doesn't seem to exist formal definitions for those concepts. Despite this, data can be collected that sidesteps this fact that can be useful: we can ask many people if they feel particular emotions and draw conclusions from that data. Still we must concede that a shared definition may not exist, which weakens these conclusions.
In the case of consciousness, we could ask many people if they thought that someone/something else was conscious, but it's not clear what value that would have.