AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency

In its infancy I think that any for public consumption AI that is allowed to interact with humans and/or external systems that can yield any external outcomes should be held liable and accountable by the owner. "In the box" or otherwise.

While those that stand to make sweeping profits will disagree - if an OpenAI product is leveraged to create a virtual friend platform, and based on how the constraints are built (or not), the publisher should be responsible if that virtual friend convinces someone to commit suicide no different than those held accountable in the physical world.

If there is no accountability we'll go right back down the path of recent history where there are no repercussions and a weak apology by will count as enough. Profits will continue to be the overall driver. Those corporations will, once again, argue that self-policing is the only way. And they will continue to brush off gross negligence by hiding behind hoards or layers and lobbyists.

I believe Eliezer uses this as a definition of intelligence, but it also works as a definition of agency: to paraphrase, a system that acts to attain or impose a pattern, criterion or constraint on future states of the world.

A crucial metric, IMO, is the degree to which paths of action to these criteria extend from the agent. For instance, a Spot robot acts to maintain the criterion of upright position on its future stance by action of leg movement. This only affects the robot directly through short plans and so is relatively harmless. In comparison, an ASI running on an AWS datacenter may impose the criterion that the datacenter continue to exist, through long chains of action involving the eventual death of all humans intending or posing a chance to destroy it. That would obviously be quite a lot worse, but I think the example illustrates how "imposing a criterion onto the future" captures the essence of agentic behavior at various levels of power and danger, without pulling in any unnecessary detritus such as "consciousness", "will" or "emotion".

Correct, and it's sad as to how few people, especially within the ML research community, ever considered how blatently easy it is to embody agents.

Literally go open ChatGPT now and ask it give you a time between X and Y where it will respond if not prompted again. Ask chatGPT to Write some kind of code that parses the tidbit at the end indicating when it will next respond, and have it blank prompt it if it is not responded to within the time limit. Boom, embodied agent with a few prompts and a tiny bit of extra code.

GPT is not general intelligence. It cannot reliably follow instructions. It cannot reliably do math. It cannot reliably do anything. It can do things well enough to trick people like the author into thinking it has general intelligence.

GPT-4 is not quite AGI. Until it can build a functional code base for an entire distributed web platform based only on business requirements, debug it’s mistakes, it can’t be AGI. It is, to perhaps coin a term, AGK, artificially generally knowledgeable. As a language model trained on an absolutely colossal dataset, it’s basically just a giant snapshot of human knowledge taken in superposition. Sure it’s probably at least 90% of the standard knowledge, but intelligence is a different thing.

I also think agency is wrapped up in AGI. Intentions & thoughts are meaningless until acted upon. Agency is not all or nothing either; Stephen Hawking had multiple augmentations, community and technological, which allowed him to continue to impact the world of physics After he lost his god given agency.

> GPT-4 has nearly aced both the LSAT and the MCAT. It’s a coding companion, an emotional companion, and to many, a friend. Yet it wasn’t programmed to be a test taker or a copywriter or a programmer. It was just programmed to be a stochastic parrot.

I disagree, it was absolutely trained to be a test taker. It’s been a second since I read the original GPT paper but there’s literally a multiple choice auxiliary learning task, where they use a separator token-embed to organize "question, context, options a, b, and c". As far as being a friend to many, is there evidence of this? I tried to talk to ChatGPT about some emotional problems to see if it was a cheap therapist, and I got flagged.