Measuring progress toward AGI: A cognitive framework

Would you also say that you cannot "train" intelligence?

I would agree that generally, purely acquiring knowledge does not increase intelligence. But I would also argue that intelligence (ie your raw "processing power") can be trained, a bit like a muscle. And acquiring and processing new knowledge is one of the main ways we train that "muscle".

There's lots of examples where your definition of intelligence (intelligence == raw processing power) either doesn't make sense, or is so narrow that it becomes a meaningless concept. Let's consider feral children (ie humans growing up among animals with no human contact). Apparently they are unable or have trouble learning a human language. There's a theory that there's a critical period after which we are unable to learn certain things. Wouldn't the "ability to learn a language" be considered intelligence? Would you therefore consider a young child more intelligent than any adult?

And to answer your question, whether learning about atoms makes you more intelligent: Yes, probably. It will create some kind of connections in your brain that didn't exist before. It's a piece of knowledge that can be drawn upon for all of your thinking and it's a piece of knowledge that most humans would not figure out on their own. By basically any sensible definition of intelligence, yes it does improve your intelligence.

Yes, but we have a choice about whether the AGI is an a$$h0l3 or not. That's the difference here. You do see that right?

This is a tricky topic to navigate because from a materialist perspective consciousness is the side effect of biochemical mechanisms. And many will point to the brain as the obvious container of our consciousness as a bullet to the head versus the arm would demonstrate.

But if a brain/intelligence is all you need to prove consciousness, then would an effectively complex set of neural networks that contained the same amount of neurons as a human be considered "conscious"? My guess is even at that level, probably not. Algorithms alone may mimic consciousness, but it won't be true consciousness.

Imagien this: what if consciousness is closer to something like the movie Avatar? What if the body our consciousness inhabits is closer to that of inhabiting a machine or computer that coexisted with the physics of the universe our body exists?

This would mean Jake from Avatar could theoretically inhabit not just a Na'Vi body, but what if they reproduced the Pandora equivalent of a squirrel for Jake to insert his consciousness into? Jake the Squirrel would be only as capable of expressing itself as the constraints of the body would allow it to.

Many religions discovered a long time ago that this is the most likely model of what we understand to be consciousness/sentience.

I'm not saying you're wrong, this is a conversation larger than what we may believe and touches into the core of what makes us humans that machine alone cannot replicate.

If you actually have a way to fully, without reductions, simulate matter, that's probably a Nobel prize coming your way.

The common scientific understanding is that this is not possible, at least not without extreme amounts of energy and time.

The dimensionality, or complexity if you'd prefer, of your logic gates is quite different from the cosmos. You might not agree but in my parlance a linear and a fractal curve are fundamentally different, and you can try to use linear curves to approximate the latter at some level of perspective if you want but I don't think you'll get a large audience claiming that there is no difference.

As far as I know we've also kind of given up on simulating neurons and settled for growing and poking real ones instead, but you might have some recent examples to the contrary?

Not very long ago, we thought that "life" was due to a non-material life-force thought to inhabit biological entities and thus raise what would be a biological machine to the status of living being.

The Occam's Razor-logic of looking for the simplest explanation possible leads me to the hypothesis that consciousness will similarly turn out to be an emergent property of the mechanical universe [1]. It may be hard to delineate, just as life is (debates on whether a virus is alive, etc.) but the border cases will be the exceptions.

Current research on whether plants are sentient supports this, IMO. (See e.g. "The Light Eaters" and Michael Pollan's new book on consciousness, "A World Appears".)

Meditation adds to this sense. We do not control our thoughts; in fact the "we" (i.e. the self) can be seen to be an illusion. Buddhist meditation instead points to general awareness, closer to sentience, as the core of our consciousness. When you see it that way, it seems much more likely that something equivalent could be implemented in software. (EDIT to add: both because it makes consciousness seem like a simpler, less mysterious thing, but also once you see the self as an illusion, that thing that dominates your consciousness so much of the time, it seems much less of a stretch for consciousness itself to be a brain-produced illusion.)

[1] To be clear, the fact that life turned out to not be a mystical force is not direct proof, it is an argument by analogy, I recognize that.