Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

131–140 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#131

Post by singlow »

zevyoura wrote: post #123
modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

Re: Superintelligence: The Idea That Eats Smart People

#132

Post by to3m »

apsec112 wrote: post #4

This article explicitly endorses argument ad hominem: "These people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way. Even though their arguments are irrefutable, everything in your experience tells you you're dealing with a cult. Of course, they have a brilliant argument for why you should ignore those instincts, but that's the inside view talking. Th…

OK, so I ignore Einstein, and I miss general relativity. And then what? If it's proven true before I die, then I accept it; if it isn't, or if it is and I continue to ignore it, I die anyway. And then it's 2015 and it's being taught to schoolchildren. High-school-educated people who don't really know the first damn thing about physics, like non-hypothetical me, still have a rough idea what relativity is, and the repercussions are.

Meanwhile, rewind ~100 years, and suppose you ignored the luminiferous aether. Or suppose you straight away saw Einstein was a genius? Oh, wait... nobody cares. Because you're dead.

So I'm not sure what the long-term problem is here.

Meanwhile, you, personally, can probably safely ignore people that appear to be cranks.

Re: Superintelligence: The Idea That Eats Smart People

#133

Post by cgio »

JamilD wrote: post #54
md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

The roundworm (c. elegans) only has 302 neurons, and about 7,000 synapses, but is capable of social behavior, movement, and reproduction. The entire connectome has been mapped, and we understand how many of these behaviors work without having to resort to additional ontological entities like your "third way". If this complex behavior can be explained using only 302 neurons, I have no doubt that the complexity of huma…

There is an implicit assumption there that scale is colinear with sophistication. It could be that only 5 neurons give rise to the complexity of consciousness, it could be that the number of neurons is irrelevant but rather their configuration is important. Also, there is a problem there if it explains only some of the behaviors. What happens with the rest, where are they coming from, if we have fully mapped the connectome?

Re: Superintelligence: The Idea That Eats Smart People

#134

Post by ikeboy »

If Elon Musk, Bill Gates, and Stephen Hawking etc all expressed belief in UFOs being from aliens, and you didn't know a lot about UFOs and affiliated cults, what would be the smart thing to believe?

Also,

>AI alarmists believe in something called the Orthogonality Thesis. This says that even very complex beings can have simple motivations, like the paper-clip maximizer.

Uh, no. The point of the paper clip maximizer is that it's orthogonal, not that it's simple.

>It's very likely that the scary "paper clip maximizer" would spend all of its time writing poems about paper clips, or getting into flame wars on reddit/r/paperclip, rather than trying to destroy the universe.

You know what can be made into poems about paper clips? Humans. You know what can have better flame wars than humans? Our atoms, rearranged into the ideal paper clip flame war warrior.

>The assumption that any intelligent agent will want to recursively self-improve

That's not really a premise. A better version would be "a likely path to super-intelligence will be a self-improving agent".

>It's like if those Alamogordo scientists had decided to completely focus on whether they were going to blow up the atmosphere, and forgot that they were also making nuclear weapons, and had to figure out how to cope with that.

Yudkowsky has argued that more should be invested in research into AI risk. There are tens of billions of dollars being spent on AI R&D, and somewhere in the tens of millions range spent on AI risk research. Even if advocates wanted us to spend hundreds of millions of dollars on risk research a year , that wouldn't make this criticism fair. You have a point that we shouldn't be ignoring other more important things for this, but to argue against increasing spending from 8 figures to 9 figures you need better arguments.

Re: Superintelligence: The Idea That Eats Smart People

#135

Post by okreallywtf »

md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

You could say that that 3rd way IS part of the physicalist picture already I think. As you say, there are such gaps in our understanding at this point we cannot say we've probed the depths of our brains and still found nothing.

It is like searching the ocean for something that by all accounts should be in the ocean, and deciding that perhaps the thing cannot be found, despite having large portions of the ocean still to search. It is too early to make that claim.

I'm not saying that there are not smug philosophers or scientists or just internet nerds that will declare the problem solved, clearly it is all in the brain and there is nothing magical to it but I think they would be incorrect. Modern physics is incredibly esoteric and shares more with meta-physics than what we tend to observe (which classical physics can explain satisfactorily for the most part). I don't think being physicalist is any impediment to having spirituality and and a sense of wonder about the universe and our place in it - the main difference to me is that we can't just accept a deus ex machina and call it a day. That may sound condescending but it isn't a choice for me personally.

Re: Superintelligence: The Idea That Eats Smart People

#136

Post by ajamesm »

ajamesm wrote: post #31

> But of course we know that there are all kinds of configurations of matter, like a motorcycle, that are faster than a cheetah and even look a little bit cooler. yet more inflammatory rhetoric from the Pinboard guy

every tech worker who bought a motorcycle to be Hard and Cool just owned my karma with downvotes

but you will never be a Hell's Angel

Re: Superintelligence: The Idea That Eats Smart People

#137

Post by samatman »

mrfusion wrote: post #28

I've always wondered if the problems an intelligence solves are exponentially hard so even if we build a super intelligence it wouldn't be all that much smarter than we are. For example compare how many more cities in the traveling salesman problem a super computer can solve vs your grandmas pc. It's more but surprisingly not all that many more. What do you think of that idea?

I think the 98% of DNA we share with chimpanzees, with brains almost three times as large, suggests that isn't the case. Language appears to be a force-amplifier in intelligence, and we have no reason to believe that other such do not exist.

Re: Superintelligence: The Idea That Eats Smart People

#138

Post by notahacker »

apsec112 wrote: post #81
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

There are also nearer-term threats (although I'd likely disagree on many specifics), but I don't see how that erases longer-term threats. One nuclear bomb being able to destroy your city now doesn't mean that ten thousand can't destroy your whole country ten years down the line.

I think the point (which is addressed with Maciej's Almogadro callback near the end) is that the longer term threat being speculated about and dependent on lots of very hypothetical things being true is pretty much irrelevant in the face of bigger problems. I mean, yes, a superpower that had hard military AI could wreak a lot of havoc. On the other hand, if a superpower wants to wipe out my corner of civilisation it can do so perfectly happily with the weapons at its disposal today (though just to be on the less-safe side, the US President Elect says he wants to build a few more). And when it comes to computer systems and ML, there's a colossal corpus of our communications going into some sort of black box that tries to find evidence of terrorism that's probably more dangerous to the average non-terrorist because it isn't superintelligent.

Ultimately, AI is neither necessary nor sufficient for the powerful to kill the less powerful.

And if it's powerful people trying to build hard military AI, they probably aren't reading LessWrong to understand how to ensure their AI plays nice anyway.

Re: Superintelligence: The Idea That Eats Smart People

#139

Post by joebubna »

The thing I think everyone should be most worried about is not robots deciding to kill us, it's the economic upheaval that could result from robots that can do jobs better than their human counterparts.

There's been studies done on this type of thing (https://journalistsresource.org/studies/economics/jobs/robot...) and so far use of robots has mostly focused on helping human workers become more productive, and not replacing them entirely. However, for lower skilled workers this isn't always true, and if robots were able to replace even the most skilled workers... that could cause problems for human employment. To quote the article on current status:

"Robots had no effect on the hours worked by high-skilled workers. While the authors found that industrial robots had no significant effect on overall employment, there was some evidence that they crowded out low-skilled and, to a lesser extent, middle-skilled workers."

Now continue that line of thinking and imagine a world where a robot could do any job better than a human...

We could end up with a "is that American made?" or "is that free range chicken?" type of scenario where companies that refuse to replace human workers with robots are competing against other companies that will do anything to lower costs, even if ethically questionable.

So then we potentially end up in a situation where the rich (Executives and stock holders) get richer by replacing costly human workers with cheaper, more efficient robots, and the wealth of the average family declines as people struggle to find work. Alright, well maybe we give all humans an allowance to live off of, food to eat, a home to live in, etc. Except...

Human beings need work. They need to feel a sense of purpose. I don't think the humans from the movie Wall-E hold much appeal. Let's not go there.

Ok, so maybe we pass laws against replacing many human jobs with robots. Well, if the robots are truly intelligent, aren't we then discriminating against a group of sentient beings solely because they are too good at their job? Isn't this just going to be a techno world version of the civil rights and lgbt rights movements?

These are the things I worry about. Not robots killing me.

As a side note, I hope cyborg and other bio tech improves at some point, at lot of these concerns could be mitigated if humans had the potential to improve themselves beyond any normal evolutionary rates.

Re: Superintelligence: The Idea That Eats Smart People

#140

Post by k8t »

idlewords wrote: post #95
k8t wrote: post #72

Let's take a step back and look at the meta-level. So there's maybe a chance AI is extremely dangerous (as in wipes humanity out), and there's a chance that AI might not be dangerous. There are arguments between how likely each choice is. But the more important fact is that since there's so much at stake (all of humanity), we should likely be really really really sure about things. For example, let's say you live in…

The problem with this precautionary reasoning is it leads you to Pascal's Mugging, where you are ready to believe very unlikely things because of the enormous impact they'll have if true.

Yes, I agree with the reasoning behind Pascal's Mugging. But Pascal's Mugging refers to things that have an astronomically low chance of happening — like 0.000000001. But is the chance that AI is dangerous that low? Nobody in the world knows for sure at this point, due to how far away superAI might be, and due to the uncertainties in implementation. Therefore, if we use Bayesian thinking and spread it out, I'm not really sure you could put it below 1% (I pulled this number out of thin air, but everybody at this point is doing the same).
Post reply on HN