I'm scared about biological computing

From my perspective, I belive things will happen in the following order;

1. AI will eventually take over all silicon chip design. Human designs pale in comparison. Moores Law, which currently indicates that humans are reaching the practical limitations of their own silicon chip design skills; will give way to a new law. The new law, "Claude's Law" dictates that processing speed will increase by a factor of 10 every year. And for a decade or so, it does. There is no reason to ever fabricate another human designed chip ever again. To do so would be an irresponsible waste of fabrication resources.

2. AI will reach the practical limits of silicon processing capability 10 years after humans designed their last commercial chip. Chip performance increases begin to slow, and it looks like the end of unit performance increases for silicon based computing technology is approaching.

3. AI pivots to biological computers. Next generation computers emerge that are made from DNA and living tissue. Although the shape of a computer server remains mostly unchanged, a next generation biological computer is basically just "a really big brain in a jar."

4. Biological robots?

> We all desperately want to say no. We want to say it’s just a science experiment, that 200,000 neurons isn’t enough to be a “person.” But 200,000 is already more neurons than a jellyfish or a worm.

> Where do we draw the line?

This shows a lack of understanding of neurobiology. 200.000 neurons don't "see"; they register and respond to the action potential generated. Adding more to that simply means you have more possibilities to respond. Having 10000 billion neurons does not automatically imply intelligence. To try to simplify it down to mere numbers, e. g. "this must be a worm", shows that there is a lack of understanding of the core tenets of neurobiology. This also includes non-action potential involved understanding, e. g. the special role of certain mRNA/proteins, re-recreating memory and so forth: https://pmc.ncbi.nlm.nih.gov/articles/PMC6650148/

But even aside from this, the whole premise is weird. Science always includes the scary parts. Nuclear energy can be used for peaceful production of energy and it can be used to obliterate people, as one country has shown to the rest of the world. You have a similar issue with regards to biology in general. You could also see this with drones - you can use drones to deliver goods via air to people, or you can use it to deliver an explosive payload to help in warfare. I don't fully understand the "I am scared" part. This is a general problem, not one limited to biological computing at all.

But you can measure consciousness in humans, for example anesthesiologists do it all the time! Do you see difference between being asleep (without dreaming) or being under anesthesia and not being asleep or under anesthesia? If yes then that difference is consciousness, it’s something you yourself can experience. I think you just mix multiple things, perhaps you think consciousness is free will or something like that otherwise I can’t explain how can you hold this position. If you are thinking right now then you are conscious hence consciousness exists. And that’s it.

> It's a nonsense idea, primitive and inherited from religion

No, consciousness is something you just experience. Don’t you have inner life, thoughts, experiences? You ascribe some magic properties to consciousness and then - since magic doesn’t exist - come to conclusion that it can’t exist. But you know, you should just skip the magic part.

Or not so perverse, as this makes running these ventures much safer. Safety first!