Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is there a canonical source for “the argument for AGI ruin” somewhere

twitter.com

Post reply on HN
Print view View original

41–50 of 81 posts

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#41

Post by ChancyChance »

mitthrowaway2 wrote: post #39
ChancyChance wrote: post #35

Earlier quoted context omitted.

"Concerns about superintelligence have been voiced by leading computer scientists and tech CEOs such as Geoffrey Hinton,[6] Alan Turing,[a] Elon Musk,[9] and OpenAI CEO Sam Altman." Oh my, the implications of this sentence are staggering. Equating Musk on the same level of Turing is the definition of a juxtaposition, but that's not even my point: the second juxtaposition is putting inventors of computing theory next…

It's a wikipedia article, and that sentence is intended to establish that the topic is notable: that "serious people take it seriously". It needs to establish this in the minds of a very diverse group of readers. For some readers, Alan Turing is a serious person whose opinion matters; for others, Elon Musk is a serious person whose opinion matters; for others, nobody's opinion matters except their own.

That's a really good explanation I hadn't thought of: an example of deploying both-side-ism when both-side-ism is desperately needed. Shrewd, but I wonder if it was intentional or just the outcome of an editing war of egos.

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#43

Post by Cardinal7167 »

Swizec wrote: post #37
Cardinal7167 wrote: post #12

Earlier quoted context omitted.

This is what social media’s real damage is. It turned academic progress into arguable points. Your research degree and countless lab hours is equally valid to my two-second no-research hot take, and if you’re not on board with it then you’re swarmed.

> my two-second no-research hot take This is more like watching an argument between Feynman and Oppenheimer. They look like hot takes on twitter, but these guys have spent plenty of time thinking about it. Yudkowsky is an AI ethics and safety researcher, also founder of LessWrong, an HN favorite – https://en.wikipedia.org/wiki/Eliezer_Yudkowsky And Chalmers is a philosopher focusing on consciousness and cognitive sci…

I didn’t mean my comment about this tweet exchange as an example, I was only trying to speak to the comment I replied to about a general trend. This particular exchange is not the problem; it’s the inability for the public to gauge when it’s not like this, though, if that makes sense.

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#44

Post by danbmil99 »

Personally I'm a bit tired of Yudkowsky's domination of this narrative. He's right about one thing, no one is a perfect predictor. There really is no way to know in advance how this is going to play out, no matter how many thought experiments you undertake.

The doom scenarios are of course vaguely plausible, but I don't trust anyone's percentages. It's a real unknown unknown.

Personally, I suspect it's inevitable that if we create true agi, it will come to dominate the space of intelligent conscious beings on Earth. attempts to corral it, rein it in, bend it to our will and make it our tool seem bound to fail. But then this is just me prognosticating, and I don't have the platform that he has.

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#45

Post by adfgadfhionio »

dvzk wrote: post #9

I admit I’m completely out of my depth when it comes to this field — I don’t even typically care about AI — but Eliezer’s response looks really bad to anyone in research. Asking for a citable and thorough written argument is as basic a requirement as it gets. To repudiate that request with “everyone has a different objection” is nearly unthinkable. And to David Chalmers no less! I think podcast hosts, tweet authors,…

This field has created a lot of writings but, as far as I can tell, no knowledge. The subject matter of this field does not exist. I don't think you need to know much of anything to match the supposed experts. Watching Terminator is probably good enough.

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#46

Post by reducesuffering »

nradov wrote: post #14
zarzavat wrote: post #7

Earlier quoted context omitted.

AGI is a two dimensional space. The first axis is intelligence, i.e. ability to reason, learn and synthesize, ranging from below-human, human parity, and super-human. The second axis is cost. Ranging from high-cost (more expensive to run an AGI than a human), parity (costs the same as a human employee) and low-cost (much cheaper than a human employee). Different points in this space lead to different outcomes. For ex…

You can't possibly predict the outcomes with any accuracy. This is total speculation.

Exactly. It is like ants predicting humans inventing iPhones

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#47

Post by Baeocystin »

mitthrowaway2 wrote: post #31
Baeocystin wrote: post #28

Not-joking answer: James Cameron. Before him, Stanley Kubrick. Our cultural headspace has been primed to see killer robot AI, so we see killer robot AI. Which is not to downplay AGI risk per se- it's a powerful tool, and powerful anything can be dangerous. But the uniquely foomy paperclip maximizer fear? That's a cultural attractor from the bay area through and through.

Are you sure such movies weren't just written because a plausible premise is more compelling than an implausible one?

I am certain that perceptions of AI/robotic danger are more cultural, emotional reactions that reasoned ones, if that helps. Some papers for your perusal:

Cross-Cultural Differences in Comfort with Humanlike Robots: https://link.springer.com/content/pdf/10.1007/s12369-022-009...

Culture and Attitude Towards Robots: https://eprints.mdx.ac.uk/25209/1/JNS_Review%20Manuscript_Re...

These are just a couple of quick links I found that weren't behind paywalls, to illustrate how culturally-bound these types of perceptions are.

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#48

Post by tivert »

photochemsyn wrote: post #17

It depends on what is meant by 'ruin', doesn't it? For example, imagine an AI system capable of generating its own capital base via clever investing over time. It's plausible that a correctly configured and trained AI system could achieve this goal. Now, what if that same AI also started a new industrial corporation that had no shareholders or and no board of directors? Now imagine the AI does this in collusion with…

> It depends on what is meant by 'ruin', doesn't it? For example, imagine an AI system capable of generating its own capital base via clever investing over time. It's plausible that a correctly configured and trained AI system could achieve this goal. Now, what if that same AI also started a new industrial corporation that had no shareholders or and no board of directors?

> Now imagine the AI does this in collusion with the employees of this new corporation...

> This might cause 'social destabilization' by entirely eliminating the current system of investment capitalism.

That particular scenario doesn't ring true to me, because it seems to assume that that "AI systems" that sophisticated could be monopolized by those weird partnerships.

I think it's far more likely, in the case of AI, that "the current system of investment capitalism" will have access to better AI systems of better quality than anyone else (except perhaps some militaries), because they the money to access the best resources (both equipment and talent).

AGI is most likely not going to be developed by some wizard in a garage, who will then have time to let it incubate and amass power outside of existing structures. Even if some wizard manages the first part, existing power structures will likely get there soon after, following the same prior work.

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#49

Post by p-e-w »

Cardinal7167 wrote: post #12
dvzk wrote: post #9

I admit I’m completely out of my depth when it comes to this field — I don’t even typically care about AI — but Eliezer’s response looks really bad to anyone in research. Asking for a citable and thorough written argument is as basic a requirement as it gets. To repudiate that request with “everyone has a different objection” is nearly unthinkable. And to David Chalmers no less! I think podcast hosts, tweet authors,…

This is what social media’s real damage is. It turned academic progress into arguable points. Your research degree and countless lab hours is equally valid to my two-second no-research hot take, and if you’re not on board with it then you’re swarmed.

> This is what social media’s real damage is. It turned academic progress into arguable points.

If that is "damage", the problem must surely lie with academia, not with social media.

Academia's only raison d'être is to facilitate formal discourse. Wherever it doesn't do that, it loses its legitimacy. "I'm a researcher and you are not" is not an argument. In fact, I'd go so far as to claim that anyone who makes such an "argument" should be excluded from any future discourse.

Re: Is there a canonical source for “the argument for AGI ruin” somewhere

#50

Post by ChatGTP »

ilaksh wrote: post #13
uh_uh wrote: post #8

Earlier quoted context omitted.

I think you're talking about ASI (artificial superintelligence). AGI just means that it's a general problem solver as opposed to a narrow AI. It doesn't have to be superhuman.

That's what I think it should mean, but I see very few discussions these days where anyone uses AGI to describe something that is not superhuman and all of those other traits. So trying to use it as "general problem solver" which makes more sense to me also, is going to confuse most people. Part of the problem is that people don't understand that there can be a difference between general purpose and "living conscious…

That's what I think it should mean, but I see very few discussions these days where anyone uses AGI to describe something that is not superhuman and all of those other traits. So trying to use it as "general problem solver" which makes more sense to me also, is going to confuse most people.

Because almost everyone including top researches assume that once an AGI is a thing, it will quite rapidly become an ASI.

Post reply on HN