> Why we should live in such a universe is a good question, but I don’t see how that can be answered now. His last point here is salient. Much like the various interpretations of quantum mechanics, any theories of consciousness are untestable. But my problem with "consciousness" is that it's not even a well-defined concept. Suppose I replace all occurrences of "consciousness" in that article with the word "qualgma".…
I believe Koch is conflating many distinct concepts - as most of the consciousness research field seems to.
How can you say anything useful until you define your terms?
It seems to me humans have many different layers of experience:
Visceral pain/pleasure Instinctive drives Emotional/limbic experiences Locational (physical mapping) Memory-based/learned Abstract/intellectual/symbolic
Consciousness is another kind of experience which adds a symbol that works a bit like JavaScript's 'this', and seems to create a level of experiential indirection - so instead of sensing something directly, you experience how it affects the 'this' symbol. It seems to do this by abstracting the experience and simultaneously comparing it with an associative database of previous experiences, using all of the circuits in the list. (And probably others too - I won't pretend this is a complete model.)
You need some basic ability to abstract to have a 'this.' The better you are at it, the more bandwidth passes through the experience/memory/abstraction/future prediction/new memories system.
Simpler animals have basic input circuitry, and they learn instinctively without abstraction.
But... I'd guess surprisingly many animals of all kinds have a 'this' network. The human advantages are a 'this' network with more bandwidth, a much improved ability to learn from experiential invariants, an ability to make useful predictions, and an ability to store and communicate 'this' experiences using external technologies.
If this sketch is accurate, consciousness doesn't have to be mysterious. But it also doesn't depend on a simple connection count.
The Internet won't be conscious because it has no 'this' network. It has no pain/pleasure circuits, no emotional circuits, and no instinctive drives. But most of all it has no ability to compare memories with current experiences and to abstract learning from that. There are systems - like Google search - which have incredibly basic precursors to abstraction/learning/prediction. But they still mostly run on a batch basis with input->processing->output, and no explicit self-concept state memory.
My guess is that if you built a system from these elements and included the current state of the art - or better - machine learning, machine vision, and natural language elements, with useful analogs of instinctive drives and the other animal fundamentals, you'd potentially have something that at least acted and sounded conscious, and would likely do some unexpected things. (Which is one of the practical ways we think we recognise consciousness. Conscious entities are hard to model. In fact we're probably easier to model than we think we are, but face to face, consciousness in other people usually implies surprises.)
Whether or not it would be truly self-aware or just pretending is a question that's impossible to answer without resorting to metaphysics.