Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

391–400 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#391
post #70

It's amusing how his talk and Stuart Russel's talk [0] -- which end up going opposing ways -- use the "atomic bomb" as an example in fallacious thinking. Stuart's example is about how respected physicists said "impossible to get energy out of atoms" and the very next morning someone published a paper stating how to do it. [0]: https://www.youtube.com/watch?v=zBCOMm_ytwM

He and I were on a panel once! It did not go well.

There must be a video of it somewhere, no? Please! ;-)

Re: Superintelligence: The Idea That Eats Smart People

#392
post #368

Earlier quoted context omitted.

I'm not buying that. In fact, the only reason such drone attacks don't happen, or why people hadn't been casually blowing each other up for the past five decades with explosives attached to RC cars / planes is that in general, people are nice to each other . There's plenty of tools out there for dedicated people to wreck havoc in populated areas. Such people simply are very, very rare.

Also as recent news from Berlin shows, you don't even need a drone, a regular old van will do

Further backing up the parent commenter's statement that people are generally nice to each other.

Re: Superintelligence: The Idea That Eats Smart People

#393
post #368

Earlier quoted context omitted.

I'm not buying that. In fact, the only reason such drone attacks don't happen, or why people hadn't been casually blowing each other up for the past five decades with explosives attached to RC cars / planes is that in general, people are nice to each other . There's plenty of tools out there for dedicated people to wreck havoc in populated areas. Such people simply are very, very rare.

Also as recent news from Berlin shows, you don't even need a drone, a regular old van will do

[deleted]

Re: Superintelligence: The Idea That Eats Smart People

#394

Earlier quoted context omitted.

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

> High-tax Sweden has as many billionaires per capita as the US The first thing they all did was move their incorporations out of Sweden

All the more reason for leveling the playing field.

Re: Superintelligence: The Idea That Eats Smart People

#395
post #240

Earlier quoted context omitted.

We don't need ai for massive differences in military effectiveness. That is already here. The us can just destroy most country and substate actors with minimal causalities. The issue is already just difficult matters like differentiating friendlies/neutrals from enemies and not creating more enemies via collateral damage and other forms of reactions.

The problem arises when non-state actors wreak havoc with drones and AI with inpunity. What would we do if a drone made by a no-name manyfacturer dropped some bombs in Times Square? Who would we blame when someone uses AI to actually sow social mistrust and subvert our existing systems?

>The problem arises when non-state actors wreak havoc with drones and AI with inpunity.

Well, for the rest of the world, state actors wreaking havoc with drones and AI with inpunity is already a problem.

Re: Superintelligence: The Idea That Eats Smart People

#396
post #368

Earlier quoted context omitted.

I'm not buying that. In fact, the only reason such drone attacks don't happen, or why people hadn't been casually blowing each other up for the past five decades with explosives attached to RC cars / planes is that in general, people are nice to each other . There's plenty of tools out there for dedicated people to wreck havoc in populated areas. Such people simply are very, very rare.

Also as recent news from Berlin shows, you don't even need a drone, a regular old van will do

Note that the same thing happened in France recently.

Re: Superintelligence: The Idea That Eats Smart People

#397

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

No, because their power is limited by their role in society and the organization in which they operate. Even the president of the United States has limited powers. If Obama just decided one day that the best course of action was to nuke Moscow and doubled down on doing so it's extremely unlikely he'd be able to do so. There are enough other people with careers, jobs, pension plans and common decency between him and actually launching missiles that I don't think it's credible that it would happen on a whim like that, even a persistent one. Someone would call a doctor and get the President some medication.

However, that depends on the people between the president and nuclear launch being decent human beings that care about law and order and proper procedure. You need to look at the system, not just individuals.

Alternatively, let's say someone came to power who genuinely believed nuking Russia was a good option. Rather than order a launch on the spot, what they'd actually do is gradually build a case, appoint pliable or similarly thinking people to key positions, get the launch protocols revised, engineer a geopolitical crisis by provoking Russia and drive events towards a situation in which a nuclear launch seems like a legitimate option.

Re: Superintelligence: The Idea That Eats Smart People

#398

>Eventually it gets to a near-superhuman level, where it's funnier than any human being around it. >>My belt holds up my pants and my pants have belt loops that hold up my belt. >>What's going on down there? >>Who is the real hero? I love that he used a Mitch Hedberg joke for this.

No, no, no. Belt loops hold belts down ;)

Re: Superintelligence: The Idea That Eats Smart People

#400
post #326

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

People are worried about AI risk because ensuring that the strong AI you build to do X will do X without doing something catastrophic to humanity instead is a very hard problem, and people who have not thought much about this problem tend to vastly underestimate how hard it is. Whatever goals the AI has, it will certainly be better at achieving them if it can stay alive. And it will be more likely to stay alive if th…

> to do X will do X without doing something catastrophic to humanity instead is a very hard problem

This scenario I agree with. For instance: the AI decides that it doesn't want to live on this planet and consumes our star for energy, or exploits our natural resources leaving us with none.

The whole AI war scenario is highly unlikely. As per the article, the opponents of AI are all regarded as prime examples of human intelligence - many of them have voiced opposition to war and poverty (by virtue of being philanthropists). Surely something more intelligent than humans would be even less inclined to wage war. Furthermore, every argument against AI posits that humans are far more important than they really are. How much time of day do you spend thinking about bacteria in the Marianas Trench?

> AI control

My argument is with exception to this scenario. By attaching human constraints to AI, you are intrinsically attaching human ideologies to it. This may limit the reach of the superintelligence - which means that we create a machine that is better at participating in human-level intelligence than humans are. Put simply, we'd plausibly create an AI rendition of Trump.

Post reply on HN