Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How AGI became the most consequential conspiracy theory of our time

technologyreview.com

Post reply on HN
Print view View original

61–70 of 99 posts

Re: How AGI became the most consequential conspiracy theory of our time

#61

Post by JohnMakin »

I largely agree with much of this, but would like to pose the following hypothetical:

Let's say the AGI true believers, the really big players out there believe 2 things simultaneously that seem impossible together:

1) AGI will revolutionize humanity and propel us to unimaginable progress

2) AGI will destroy humanity

What if they actually believe 2) is a precursor to 1)? A lot of them seem to be building bunker fortresses on private islands right now. It seems like there is a certain class of very rich person that thinks yes, this current iteration of civilization is doomed (whether you believe it to be nuclear war, climate change, AGI, whatever), but, by virtue of them being Very Smart and Wealthy Human Beings, they can ride that out, and create a new civilization built on what was left behind.

I've not deep dove into a lot of writings by these guys but this kind of attitude seems to ooze from it. They're not concerned, because they think they're the ones that won't be affected. I think history says otherwise - when civilizations undergo catastrophic events, it tends to be the minority ruling classes that get eaten alive, not the other way around. I guess we'll see either way, right or wrong.

Re: How AGI became the most consequential conspiracy theory of our time

#62

Post by SpicyLemonZest »

AlexandrB wrote: post #37

One thing that struck me recently is that LLMs are necessarily limited by what's expressible with existing language. How can this ever result in AGI? A lot of human progress required inventing new language to represent new ideas and concepts. An LLM only experience of the world is what can be expressed with words. Meanwhile, even a squirrel has an intuitive understanding of the laws of gravity that are beyond the abi…

> Meanwhile, even a squirrel has an intuitive understanding of the laws of gravity that are beyond the ability of an LLM to ever experience because it's stuck in a purely conceptual, silicon prison.

I just don't think that's true. People used to say this kind of thing about computer vision - a computer can't really see things, only compute formulas on pixels, and "does this picture contain a dog" obviously isn't a mathematical formula. Turns out it is!

Re: How AGI became the most consequential conspiracy theory of our time

#63

Post by tim333 »

mateo411 wrote: post #57
labrador wrote: post #44

Ray Kurzweil and his "Age of Spiritual Machines", which I read in 1999, is much more to blame than the others like Goertzel that came after him but Kurzweil doesn't get a mention. Kurzweil is also a MIT grad closely associated with MIT and possibly the MIT Technology Review.

Yes, the Singularity came from Kurzweil.

Well, it predated Kurzweil. The term first came up with John von Neumann in the 50s. But Kurzweil has promoted it.

Re: How AGI became the most consequential conspiracy theory of our time

#64

Post by tim333 »

The article is kind of silly like someone in the 1800s saying flying machines are a conspiracy theory. I mean obviously not - they were a proposed technology that hasn't been built yet but would in a while. I don't really get the point. I guess it's a discussion point of a sort. Maybe it's like those Weekly World News 'Gay Aliens Found' type headlines that get views for imaginative silliness?

Re: How AGI became the most consequential conspiracy theory of our time

#65

Post by xanderlewis »

tim333 wrote: post #64

The article is kind of silly like someone in the 1800s saying flying machines are a conspiracy theory. I mean obviously not - they were a proposed technology that hasn't been built yet but would in a while. I don't really get the point. I guess it's a discussion point of a sort. Maybe it's like those Weekly World News 'Gay Aliens Found' type headlines that get views for imaginative silliness?

But flying machines are well defined, or at least it's easily possible to come up with a good definition. 'A machine capable of transporting a person from a to b without touching the ground at any point in between', or whatever.

For AGI, that's very far from being true.

Re: How AGI became the most consequential conspiracy theory of our time

#66

Post by [deleted] »

tim333 wrote: post #64

The article is kind of silly like someone in the 1800s saying flying machines are a conspiracy theory. I mean obviously not - they were a proposed technology that hasn't been built yet but would in a while. I don't really get the point. I guess it's a discussion point of a sort. Maybe it's like those Weekly World News 'Gay Aliens Found' type headlines that get views for imaginative silliness?

[deleted]

Re: How AGI became the most consequential conspiracy theory of our time

#67

Post by Terr_ »

scarmig wrote: post #40
Terr_ wrote: post #36

Earlier quoted context omitted.

> if you assume that AGI is fake and impossible That seems like a bad straw-man for "AI boosterism has the following hallmarks of conspiratorial thinking". > offer some reasoning that it's impossible Further on, the author has anticipated your objection: > And there it is: You can’t prove it’s not true. [...] Conspiracy thinking looms again. Predictions about when AGI will arrive are made with the precision of numero…

If it makes you angry that people want to work to build AGI--people who have thought about it a lot more than you--you can't convince them to stop by repeatedly yelling "I don't think it's possible, you're a fool!" No more than yelling "electricity is conspiracy thinking/Satan's plaything!" repeatedly would have stopped engineers in the 19th century from studying and building with it.

> If it makes you angry that people want to work to build AGI

What's this, a second straw-man? So quickly after the first?

TFA never condemned invention or hard work, nor does it agree with the "doomers" who consider the target-invention to be fundamentally bad. At most it's a critique of a set of beliefs/rationalizations plus choices made by investors.

> makes you angry [...] people who have thought about it a lot more than you [...] repeatedly yelling [...] "you're a fool!"

Who's angry? Who's making it personal?

I think reading the article made you angry... and you're projecting it onto everybody else.

Re: How AGI became the most consequential conspiracy theory of our time

#68

Post by chubot »

labrador wrote: post #44

Ray Kurzweil and his "Age of Spiritual Machines", which I read in 1999, is much more to blame than the others like Goertzel that came after him but Kurzweil doesn't get a mention. Kurzweil is also a MIT grad closely associated with MIT and possibly the MIT Technology Review.

Yeah totally, not a single mention of Kurzweil in this article. I also read “Age of spiritual machines” in 1999 (in college), and skimmed most of his subsequent books

Then Kurzweil became my manager’s peer at Google in 2014 or so (actually 2 managers). I remember he was mocked by a few coworkers (and maybe deservedly so, because they had some mildly funny stories)

So I have been wondering with all the AGI talk why Kurzweil isn’t talked about more. Was he vindicated in some sense?

I did get a partial answer - one reason is that doomer AGI prophecies are better marketing than Kurzweil’s brand of AGI, which is about merging with machines

And of course both kinds of AGI prophecies are good distractions from AI ethics, which is more likely to slow investment than to grow it

Re: How AGI became the most consequential conspiracy theory of our time

#69

Post by labrador »

chubot wrote: post #68
labrador wrote: post #44

Ray Kurzweil and his "Age of Spiritual Machines", which I read in 1999, is much more to blame than the others like Goertzel that came after him but Kurzweil doesn't get a mention. Kurzweil is also a MIT grad closely associated with MIT and possibly the MIT Technology Review.

Yeah totally, not a single mention of Kurzweil in this article. I also read “Age of spiritual machines” in 1999 (in college), and skimmed most of his subsequent books Then Kurzweil became my manager’s peer at Google in 2014 or so (actually 2 managers). I remember he was mocked by a few coworkers (and maybe deservedly so, because they had some mildly funny stories) So I have been wondering with all the AGI talk why Ku…

> Was he vindicated in some sense?

No. He's still saying AGI will demand political rights in 2029. Like Geoffrey Hinton, Kurzweil gets a pass because he's brilliant and acomplished. But also like Hinton, he's wrong about this one issue. With Hinton it appears to be fear driving his fantasies. With Kurzwel it's probably over-confidence.

Re: How AGI became the most consequential conspiracy theory of our time

#70

Post by XenophileJKO »

AlexandrB wrote: post #37

One thing that struck me recently is that LLMs are necessarily limited by what's expressible with existing language. How can this ever result in AGI? A lot of human progress required inventing new language to represent new ideas and concepts. An LLM only experience of the world is what can be expressed with words. Meanwhile, even a squirrel has an intuitive understanding of the laws of gravity that are beyond the abi…

I don't know why you would think that the model can't create new language. That is a trivial activity. For example, I asked GPT5 to read the news and make a new word.

Wattlash /ˈwɒt-læʃ/

n. The fast, localized backlash that erupts when AI-era data centers spike electricity demand—triggering grid constraints, siting moratoriums, bill-shock fears, and, paradoxically, a rush into fixes like demand-response deals, waste-heat reuse, and nuclear/fusion PPAs.

Post reply on HN