CEO fired developers to make room for AI. Developers create open source AI CEO

the main job of a tech CEO these days is to 1) raise money, 2) keep the board happy.

I don't see either of those being able to be done by an AI agent.

Because what I meant by that of course was power over money. You turn it on, put in the credit card info and let it rip, eh?

Then the board can fire that CEO?

Thanks for the reply. I was hoping to get some debate going and have my thoughts challenged in order to see how they hold up under scrutiny. You're right that I believe in Functionalism and Strong AI and in previous comment I only really stated my belief so please allow me to try and make a bit of an argument for it. My position probably falls under the "System reply" to the Chinese Room argument although I feel it's more restrictive in some aspects so perhaps it corresponds to a particular subtype (I don't know as I haven't read enough in the area).

My basic argument is that human *intelligence* (and understanding - I consider understanding as part of intelligence and consciousness separately but I can see how many might group understanding with consciousness and leave intelligence on its own) is a form of collective intelligence. Individual neurons acting based on local chemical and electrical signals collectively produce behaviour that we call intelligent, i.e. it can solve problems, respond to questions, etc ... . A particular configuration of neurons acting together I will call a particular software or algorithm (I understand that separating the software from the neural substrate in that way is part of what's in dispute, particularly by Searle, but that's my belief and what I feel is required to make the mapping to the Chinese Room argument). The Chinese Room thought experiment then is trying to refute Strong AI by arguing that the person in the Chinese Room has no understanding of the exchange with the outside world because they are just manipulating symbols incomprehensible to them. I agree that the person has no understanding. That person on their own would not be able to string together a sensible reply without the instruction cards. However another person with the same instruction cards, i.e. algorithm, would be able to reply sensibly, pass a Chinese Turing test, etc ..., thus demonstrating that the intelligence/understanding resides in the algorithm. The person is just an execution substrate. The mistake in the Chinese Room argument is identifying the locus of understanding in the person, which imho is failure of imagination. It's akin to saying that humans can have no understanding because each individual neuron has no understanding of our actions, something which the existence proof of human intelligence refutes.

What then of consciousness? Here I also have some beliefs which I'm happy to write down but in my assessment they're less substantiated so make of them what you will. I saw some things by Joscha Bach recently and I liked what he said about agents and consciousness as a form self-recognition of the agent as an entity in the environment and predictive modeling of future states of that entity in order to achieve the agents goals and realise some form of agency. I don't know where neuroscience and cognitive science stand on this but made sense to me as a sensible model of consciousness. Therefore I would even go so far as to say that I believe that entities such as the United States of America (and any country or institution really) are probably conscious in some form, because they are aware of themselves, model themselves, and try to project their agency in the environment that they find themselves in. We are not directly aware of this because we are like single neurons in our brain and this happens at a scale much larger than us. Moreover I suspect that as you go up each rung on the emergence ladder, the temporal scale of the phenomena gets an order slower as that level is composed of the interactions at the level below it. So the speed of interactions of our neurons is an order faster than our conscious processes and similarly the emergent bodies and institutions that we form part of act an order slower so their processes are more difficult for us to perceive.

Schematically then my beliefs organise roughly as follows:

- intelligence = algorithm: think neural network weights you can download of HuggingFace

- consciousness = self-aware world model + state: I speculate that the same NN world model weights running with separate state would perceive themselves as separate entities

Would love to hear some thoughts on this!

I kinda agree that the financial system is intelligent in the problem solving sense but disagree that it is pursuing a particular goal in the sense that institutions and organisations are.

It's more akin to an ecosystem to me. In nature we also have ecosystems organising and reusing resources but I would only ascribe intelligence to living organisms pursuing individual goals with agency.

Haven't thought too hard about the distinction though so happy to be shown a better principle.