Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

291–300 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#291

Post by nl »

ajamesm wrote: post #113
apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

The ACA could barely roll out a website without tragic failure, how have cell phones "dramatically improved" the lives of the poor? They're still subject to as much bureaucracy and denial of basic services as ever. 4G hasn't improved public transpo. I suppose the poor are no longer subject to long-distance fees during daytime.

http://www.diva-portal.org/smash/get/diva2:205909/fulltext01...

http://www.ictworks.org/2016/06/27/yes-farmers-do-use-mobile...

Etc. And that's just easily measured benefits. There's good reasons why more people in Africa have access to cell phones than to clean water.

Re: Superintelligence: The Idea That Eats Smart People

#292

Post by jessup »

>AI alarmists are fond of the paper clip maximizer, a notional computer that runs a paper clip factory, becomes sentient, recursively self-improves to Godlike powers, and then devotes all its energy to filling the universe with paper clips.

>It exterminates humanity not because it's evil, but because our blood contains iron that could be better used in paper clips.

The consolation of the cingularity narrative is that it tells a story in which the world is not—yet—being destroyed by paper clip maximizers—in cingularity land, the nightmare scenario is still a ways off, and thankfully, with the help of a few Ayn Rand types, market forces can still save us.

Re: Superintelligence: The Idea That Eats Smart People

#293

Post by fjdlwlv »

jzwinck wrote: post #157

> an artificial intelligence would also initially not be embodied, it would be sitting on a server somewhere, lacking agency in the world. It would have to talk to people to get what it wants. Organized crime and semi-organized criminal gangs stand to establish a highly effective symbiosis with amoral machines which "lack agency." If a machine wants to kill someone, all it needs to do is find a person to carry out th…

But why ever would a machine want to kill someone? Only if someone turned the machine on to some reason. That person has agency . And you don't nee SuSuperintelligence in order for a drone to bomb the wrong person, or for a computer virus to destroy the internet, or a biochemical virus, or any other self replicating disaster.

Re: Superintelligence: The Idea That Eats Smart People

#294

Post by rl3 »

The notion that near-term AI concerns and existential AI concerns somehow represent a binary option that we must choose between is fallacious at best.

Near-term AI concerns represent a massive challenge encompassing many ethical and social issues. They must be addressed.

Existential AI concerns, while low probability, have consequences so dire that they warrant further research regardless. These too must be addressed.

There is ample funding and human resources to work on both problems effectively. Why fight about it?

Re: Superintelligence: The Idea That Eats Smart People

#295

Post by kazagistar »

tachyonbeam wrote: post #225
kazagistar wrote: post #189

They always miss a critical and subtle assumption: that intelligence scales equal to or faster then the computational complexity of improving that intelligence. This is the one assumption I most skeptical of. In my experience, each time you make a system more clever, you also make it MUCH more complex. Maybe there is not hard limit on intelligence, but maybe each generation of improved intelligence actually takes lon…

I think the issue is that once do manage to build an AI that matches human capabilities in every domain, it will be trivial to exceed human capabilities. Logic gates can switch millions of times faster than neurons can pulse. The speed of digital signal also means that artificial brains won't be size-limited by signal latency in the same way that human brains are. We will be able to scale them up, optimize the hardwa…

But making them linearly faster to scale them up doesn't help if the difficulty of the problems they face isn't linear. If it comes to making more clever things, I strongly doubt they are even a remotely small polynomial.

Re: Superintelligence: The Idea That Eats Smart People

#296

Post by jimmcslim »

Galaxeblaffer wrote: post #146
jimmcslim wrote: post #36

Earlier quoted context omitted.

Any recommendations on decent pop-sci books that explore the latest understanding of the origin and mechanism of consciousness? Is anyone exploring this third way at all?

I will probably be down voted for this.. But I enjoyed Kurzweil's "How to create a mind" and "on intelligence" by Jeff Hawkins . Even though these books aren't super scientific i really like how these guys think about consciousness and intelligence.

Thanks, "On Intelligence" has been on my reading list for a while after seeing him give a keynote a few years ago.

Re: Superintelligence: The Idea That Eats Smart People

#297

Post by ikeboy »

rspeer wrote: post #281
ikeboy wrote: post #134

If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe? Also, >AI alarmists believe in something called the Orthogonality Thesis. This says that even very complex beings can have simple motivations, like the paper-clip maximizer. Uh, no. The point of the paper clip maximizer i…

> That's not really a premise. A better version would be "a likely path to super-intelligence will be a self-improving agent". If I understand your claim, it's not very relevant. You seem to be saying that, given that superintelligence happens, the probability that it will have happened via a self-improving agent is high. That doesn't refute the claim that "superintelligence probably will not happen". > You have a po…

> You seem to be saying that, given that superintelligence happens, the probability that it will have happened via a self-improving agent is high.

I'm saying that there's no need to assume that "any intelligent agent will want to recursively self-improve". It's a strawman he deliberately uses so that rejecting it lets him get a dig in at his opponents. Of course not every agent will want to improve that, and nobody relevant ever said that every agent will.

>Why don't we spend that money on non-vague problems that actually face us now? For example, unbiased AI and explainable AI would help us right now and don't require wacky far-future predictions.

I said that in response to his absurd claim that AI risk spending is taking a lot away from other things we could worry about.

But anyway, if you look at what MIRI actually produces (see https://intelligence.org/all-publications/), you'll find many papers that don't depend on "wacky" predictions.

Re: Superintelligence: The Idea That Eats Smart People

#298

Post by waqf »

jessup wrote: post #292

>AI alarmists are fond of the paper clip maximizer, a notional computer that runs a paper clip factory, becomes sentient, recursively self-improves to Godlike powers, and then devotes all its energy to filling the universe with paper clips. >It exterminates humanity not because it's evil, but because our blood contains iron that could be better used in paper clips. The consolation of the cingularity narrative is that…

From your comment I'd like to segway into a discussion about how brand names are causing people to forget how to spell normal English words.

Re: Superintelligence: The Idea That Eats Smart People

#299

Post by pfisch »

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

I mean if you made me a disembodied mind connected to the internet that never needs to sleep and can make copies of itself we would be able to effectively take over the world in like ~20-50 years, possibly much less time then that.

I make lots of money right now completely via the internet and I am not even breaking laws. It is just probable that an AI at our present level of intelligence could very quickly amass a fortune and leverage it to control everything that matters without humanity even being aware of the changeover.

Post reply on HN