Simulation, Consciousness, Existence (1998)

>A simulation is only dealing with form.

It seems to me a sufficiently precise simulation would necessarily capture meaning. If meaning is critical to decision making and that decision making is precisely simulated, then the simulation must also capture meaning.

> It seems to me a sufficiently precise simulation would necessarily capture meaning.

That is the very bone of contention here.

> If meaning is critical to decision making.

I don't believe it is in the general case. Our current crop of Go winning machines seem to indicate otherwise.

Agree, one of the better books on the subject in the sense of framing a potential path towards the robotic revolution.

It's amazing how many people speak about AGI as if it's even a theoretically plausible thing.

It's about as plausible and well substantiated as the Christian God. Which isn't to say it's impossible, but just that it's almost entirely an article of faith. There are a few anecdotes which maybe hint at its possibility. But nothing even approaching a rational explanation of how it might be built.

Also, the model of cognition you are describing (information processing or I/O) is very old, from the 1960s. It was inspired by the discovery of the computer. There are other models, like Ecological Psychology, Embodied Cognition, Distributed Cognition.

It is tempting to draw a box around the brain and posit that it's the most important interface and that all of the important information passes through it. But whenever you break down so-called "cognitive" phenomena, you find that often very little information passes through that barrier. The lions share of encodings remain outside the brain, and for any given animal task, quite a lot of information processing happens outside the brain.

In a weird way, the information processing model is a vestige of the notion of a soul. If you really accept "physical fundamentalism" as OP describes it, then the interface between the brain and the rest of the world is nothing at all. Just a ribbon of atoms with a name. No more interesting than the interface between your stomach lining and the bacteria inside, or between the vibrations in front of your mouth and all nearby ears.

The only reason to center the brain/environment interface is to try to separate what you consider to be the essential identity of a person from their physical grounding. I.e. to maintain a model which includes a soul.

The premise about computation being observer-relative overestimates how much meaning we read into a system, and underestimates how much we read out of it. When two 1kg bags of sand land on a lever and counterbalance the one 2kg bag at the other end, that has an objective, logical, correspondence-based meaning that transcends observers. When a domino computer "executes" - same. When a silicon computer computes - same. As the computer gets more complex, it becomes harder to intuit how the physical working is objectively a computation, but this proceeds by straightforward extension from the bags of sand.

We can certainly attach semantics to the numbers (e.g. saying this bit pattern represents dollars, or a spaceship's shield %), and that is observer-relative. But that is completely different from categorizing or understanding a process as a computation in general, in terms of logic.

I'll stop short of implying that it's the same for the human brain, because nobody should be pretending to understand the brain at this point in history. However, this does provide a way to see how it could be true for the brain, if it is ever determined that the brain is precisely equivalent to a computer.