Is there a canonical source for “the argument for AGI ruin” somewhere
1–10 of 81 posts
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#2Is there a universally accepted definition of AGI?
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#3Is there a universally accepted definition of AGI?
Obviously not. There is no universally accepted definition of anything. (And, I don't understand Chalmers' question. Some humans have always feared automation, back to the Luddites and probably before.)
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#4This feels like it's too broad of a question to get a good answer in any short format, but I have previously seen and will paraphrase here a better question:
.
Is there a canonical source for the argument that most of the probability space of entities that human civilization might:
(a) Qualify as AGI
(b) Cause to come into existence in the near future
corresponds to entities that would have both:
(i) goals involving the ruin of human civilization
(ii) the ability to carry out those goals?
.
This framing is better at inhibiting the people who lack significant math and/or ML knowledge from participating, which seems a priority for any public internet discussion about probability distributions over NNs and transformers.
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#5Is there a universally accepted definition of AGI?
People use it very inconsistently but I think it now for most people means something like a living digital person with god-like superintelligence although the personhood part is very fuzzy.
I always wanted it to apply to any general purpose AI including things like GPT-4, since we don't have another term for that. But in most people's fuzzy brains it connotes being alive, conscious, animal/humanlike, fully autonomous and usually they also assume it has become a million times smarter than human in a short time frame.
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#6Is there such a thing as a "canonical source" for an argument? There might be certain well-known or classical sources, but new perspectives and new chains of reasoning may be popping up all the time.
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#7Is there a universally accepted definition of AGI?
AGI is a two dimensional space.
The first axis is intelligence, i.e. ability to reason, learn and synthesize, ranging from below-human, human parity, and super-human.
The second axis is cost. Ranging from high-cost (more expensive to run an AGI than a human), parity (costs the same as a human employee) and low-cost (much cheaper than a human employee).
Different points in this space lead to different outcomes. For example parity intelligence and low cost leads to a world where knowledge workers all lose their jobs. Whereas super-intelligence and high-cost leads to a world where government entities have huge amounts of power over us. Super-intelligence and low-cost leads to a chaotic singularity.
Currently we are hovering around far below-human intelligence and very low-cost ($20/month for ChatGPT).
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#8Is there a universally accepted definition of AGI?
People use it very inconsistently but I think it now for most people means something like a living digital person with god-like superintelligence although the personhood part is very fuzzy. I always wanted it to apply to any general purpose AI including things like GPT-4, since we don't have another term for that. But in most people's fuzzy brains it connotes being alive, conscious, animal/humanlike, fully autonomous…
I think you're talking about ASI (artificial superintelligence). AGI just means that it's a general problem solver as opposed to a narrow AI. It doesn't have to be superhuman.
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#9I admit I’m completely out of my depth when it comes to this field — I don’t even typically care about AI — but Eliezer’s response looks really bad to anyone in research. Asking for a citable and thorough written argument is as basic a requirement as it gets.
To repudiate that request with “everyone has a different objection” is nearly unthinkable. And to David Chalmers no less!
I think podcast hosts, tweet authors, bloggers and live streamers sometimes forget that progress in academic fields comes from real contributions, and that public conversations (especially oral) don’t really do anything besides spread common awareness.
Re: Is there a canonical source for “the argument for AGI ruin” somewhere
#10Is there a universally accepted definition of AGI?
AGI is a two dimensional space. The first axis is intelligence, i.e. ability to reason, learn and synthesize, ranging from below-human, human parity, and super-human. The second axis is cost. Ranging from high-cost (more expensive to run an AGI than a human), parity (costs the same as a human employee) and low-cost (much cheaper than a human employee). Different points in this space lead to different outcomes. For ex…
> leads to a world where government entities have huge amounts of power over us
Aren't we already there? Because power must be ceded to government so that it can govern?