AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
secondbreakfast.co
AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
1–10 of 129 posts
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#2How does agency arise? By letting GPT-4 tell humans what to do.
With suitable prompts, it shouldn't be hard to configure GPT-4 as a boss.
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#3I think you hit the nail in the head, Will.
I’ve been reading so much o the subject (like everyone else I suppose), but you summarized my key concerns.
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#4> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization.
No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either.
An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For the first everyone seems to be rushing as fast as possible to hook up everything they possibly can to AI. For the second, we're already seeing AI do all sorts of things we didn't expect it to do.
And for the third, again everyone seems eager to create looping/recursive structures for AIs as soon as possible.
Once you have all of this, all it takes a cascade of sufficiently inscrutable and damaging reactions from the AI to do serious harm.
See e.g. https://www.lesswrong.com/posts/kpPnReyBC54KESiSn/optimality...
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#5As a long-time LLM enjoyer, by far the most insightful take I saw along these lines is https://generative.ink/posts/simulators/
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#6> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…
> An AI merely needs to be hooked up to enough physical systems
Don’t even need this. People spend quite a lot of time in virtual space. Pretending that damage there isn’t real is overlooking things. For example, the vast majority of people’s banking is done virtually and digitally. If I drain your bank account, that’s going to harm you even though I haven’t impacted you physically as I would have to with a robbery.
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#7I'm trying to make the term AGC or "Artificial General Competence" stick. Perhaps this makes me a huge arrogant asshole, but I would argue that most humans are not even competent, let alone intelligent. GPT-4, in my mind, has already surpassed the bulk of humanity in terms of competence. This milestone is (IMO) significant enough to blow up society.
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#8> Almost all AGI doomsayers assume AGI will have agency. They have this vision of the machine deciding it’s time to end civilization. No. Agency is not a necessary condition for AI to do massive damage. I don't believe agency is really well-defined either. An AI merely needs to be hooked up to enough physical systems, have sufficiently complex reaction mechanisms, and some way of looping to do a lot of damage. For th…
I actually see this as a positive thing. Rather than one bad actor hooking an AI up to a headless browser at some point in the future we are trying everything that could possibly go wrong long before the AI can do much real damage (in a technical sense, as opposed to misinformation campaigns and job replacement etc).
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#9> Almost all AGI doomsayers assume AGI will have agency
I disagree. Concern about the use of AI without agency by its human masters as a tool of both intentional and incidental repression and unjust discrimination resulting in a durable dystopia is far more common an “AI doom” concern than any involving agency.
In fact, the disproportionately wealthy and invested in AI crowd pushing agency-based doom scenarios that the media pays the most attention to are using their visibility and economic clout to distract from the non-agency-dependent AI doom concerns, and to justify narrow control and opacity which makes the non-agency-based doom scenarios (which they are positioned to benefit from) more likely.
Re: AI vs. AGI vs. Consciousness vs. Super-Intelligence vs. Agency
#10I couldn't believe it when someone posted a story about a possible AI winter approaching, and with so many comments agreeing. GPT-4 is a game-changer that's only getting started.