Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

441–450 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#441

Post by keldaris »

Ygg2 wrote: post #369
modeless wrote: post #177

Earlier quoted context omitted.

It's not that AIs will do worse things than humans have already done. It's that AIs will do those terrible things much more efficiently and with much lower risk to the people in charge.

On the plus side - no more rape.

I don't think you can hope for even that much. Rape has historically been systematically used for terrorizing the population in order to achieve military and/or political aims. An AI free from any ethical concerns could conceivably evaluate it as an efficient strategy for achieving some set of goals and proceed accordingly.

Re: Superintelligence: The Idea That Eats Smart People

#442

Post by tormeh »

apsec112 wrote: post #88
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

>If you raise the marginal tax rate to 99%, then you get rid of billionaires

No, you don't, because billionaires's source of money is almost always capital gains. They don't give a shit about income tax.

Re: Superintelligence: The Idea That Eats Smart People

#443

Post by fergie »

simonh wrote: post #397
zevyoura wrote: post #123

Earlier quoted context omitted.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

No, because their power is limited by their role in society and the organization in which they operate. Even the president of the United States has limited powers. If Obama just decided one day that the best course of action was to nuke Moscow and doubled down on doing so it's extremely unlikely he'd be able to do so. There are enough other people with careers, jobs, pension plans and common decency between him and a…

This is a really strong argument for big government. The smaller our all-powerful military command is, the easier it is for it to go rogue.

Re: Superintelligence: The Idea That Eats Smart People

#444

Post by philh »

hueving wrote: post #236
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Can you explain to me why you think taking all of the wealth of 100 billionaires will help the poor? 100 billion spread over the population of California is less than $3000/person. So you can wipe out all of the billionaires and give everyone $50/week for one year, what is that going to change?

To add some context, California's tax revenue for 2014-2015 was $130bn. So having an extra $100bn would order-of-magnitude double their income for one year.

It's not clear to me whether or not that would make a big difference. I lean no, because my default assumption is that governments are really bad at spending money, but I could see it going either way.

Of course, there are also poor people outside California, and there's no particular reason to focus on the ones inside.

(I also note that 100 billionaires own considerably more than $100bn between them, but that's a minor nitpick.)

Re: Superintelligence: The Idea That Eats Smart People

#445

Post by kabdib »

Senji wrote: post #384
naasking wrote: post #235

Earlier quoted context omitted.

> The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. To be fair, it's a small step from effective AI that doesn't malfunction, to an AI over which humans have lost control. It's precisely one vaguely specified command away in fact, and hum…

You can always use LEO EMP nukes to bring us back to the stone age, thus taking out the AI.

You don't even need to do that; just stop mining coal, or operating oil and gas fields, or scram the reactors. Or more easily, open some circuit breakers; in a pinch you can take down a few electrical towers (not many).

A rogue AI's "oxygen" is electrical power, which is really kind of fragile. The emergency power for most datacenters won't last more than a few days without replenishing diesel fuel.

For a distributed threat, take out fiber with backhoes. Happens every day now, we just happen to repair it. Stop fixing cuts and it's "Dave, my mind is going...".

Of course you have to deal with the AI making contracts with maintenance crews of its own, and do all this before it hardens its power supplies. But our current infrastructure is definitely not hardened against low to moderate effort.

Re: Superintelligence: The Idea That Eats Smart People

#446

Post by varjag »

fergie wrote: post #443
simonh wrote: post #397

Earlier quoted context omitted.

No, because their power is limited by their role in society and the organization in which they operate. Even the president of the United States has limited powers. If Obama just decided one day that the best course of action was to nuke Moscow and doubled down on doing so it's extremely unlikely he'd be able to do so. There are enough other people with careers, jobs, pension plans and common decency between him and a…

This is a really strong argument for big government. The smaller our all-powerful military command is, the easier it is for it to go rogue.

It's rather an argument for a crafted system of checks and balances than a big government per se. The totalitarian governments, for example, are absolutely massive, and many don't even have some sort of politburo to reason in a dictator.

Re: Superintelligence: The Idea That Eats Smart People

#447

Post by pjc50 »

nine_k wrote: post #246
EGreg wrote: post #240

Earlier quoted context omitted.

The problem arises when non-state actors wreak havoc with drones and AI with inpunity. What would we do if a drone made by a no-name manyfacturer dropped some bombs in Times Square? Who would we blame when someone uses AI to actually sow social mistrust and subvert our existing systems?

I'm pretty sure that drone-tracking equipment is mounted all over NYC, and more is being deployed as we talk. I can also imagine that a rouge drone arriving from afar and large enough to carry a bomb will be shot down by police if it can't be identified. If it does not carry a bomb, police will apologize.

I can hardly believe that. Any drone large enough to carry a camera can carry a hand grenade. They don't have much range but can be launched locally and flown into a crowd.

Re: Superintelligence: The Idea That Eats Smart People

#448

Post by notahacker »

FeepingCreature wrote: post #329
notahacker wrote: post #138

Earlier quoted context omitted.

I think the point (which is addressed with Maciej's Almogadro callback near the end) is that the longer term threat being speculated about and dependent on lots of very hypothetical things being true is pretty much irrelevant in the face of bigger problems. I mean, yes, a superpower that had hard military AI could wreak a lot of havoc. On the other hand, if a superpower wants to wipe out my corner of civilisation it…

That's not how risk works. If we want to grow to adulthood as a species and manage to colonize the cosmos, we need to pass _every_ skill challenge. If in 50 years there'll be a risk of unfriendly superintelligence and it'll have needed 40 years of run-up prep work to make safe, then it will do us absolutely zero good to claim that we instead concentrated on risk of military AI and hey, we got this far, right? Conside…

I think the article dealt pretty well with risk: you survive by focus finite resources on the X% chance of stopping Y% chances of the near elimination of humanity, not the A% chance of stopping a B% probability of an even worse than the near elimination of humanity event where X is large, Y is a small fraction and the product of A and B is barely distinguishable from zero, despite it getting more column inches most of the rest of the near-neglible probability proposed solutions for exceptionally low probability extinction events put together.

I also tend to agree with Maciej that the argument for focusing on the A probability of B is rescued by making the AI threat seem even worse with appeals to human utility like "but what if, instead of simply killing off humanity, they decided to enslave us or keep us alive forever to administer eternal punishment..." either.

Re: Superintelligence: The Idea That Eats Smart People

#449

Post by philh »

Apocryphon wrote: post #358
kobayashi wrote: post #317

Earlier quoted context omitted.

Of course, it does have a cheeky tone, though I think all of my points stand. The "interesting points for a layman" are actually a series of straw-man propaganda arguments. It does not argue in good faith and it should not be afforded the legitimacy of a thoughtful opposing position.

"straw-man propaganda" seems needlessly dismissive for arguments that, prima facie, don't seem completely outlandish.

If something is a straw man, you're not going to discover that by looking prima facie.

Like, I could counter this article by saying "this dude thinks evolution has made humans as intelligent as it's possible for anything to be, but there's no reason to think that's so". And prima facie, my argument isn't outlandish. Nevertheless it's a total straw man.

Re: Superintelligence: The Idea That Eats Smart People

#450

Post by tripzilch »

kenko wrote: post #238
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

> It's a military AI that correctly interprets a command to kill a particular group of people, so effectively that its masters start thinking about the next group, and the next You know, you don't need to go that far. You know what a great way to kill a particular group of people is? Well, let's take a look at what a group of human military officers decided to do (quoting from a paper of Elizabeth Anscombe's, discuss…

what are you quoting from? who's Kenny? context?
Post reply on HN