Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

411–420 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#411
What the author is actually talking about is of course the mythological techno-singularity event, which is of course bullshit.

Have you ever programmed any piece of software that suddenly implemented features of its own? Did your program become sentient?

Laughable. These are child-like fantasies belonging in 50s sci-fi.

What I do know for a fact, is that when it comes to AI fear mongers, there is not much intelligence to be detected.

Re: Superintelligence: The Idea That Eats Smart People

#412
I just love to see how my mind is so fallable. I want to believe that many of things I hold in my head are data, facts and logical theories and forget what assumptions I have made to get there. Not saying whether the author is right or not but he brilliantly points out some potential flaws and logical leaps that must be taken to get there.

Re: Superintelligence: The Idea That Eats Smart People

#413

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Don't malfunction? What are you smoking? Everything malfunctions eventually, from software to machinery to cells and neurons.

Re: Superintelligence: The Idea That Eats Smart People

#414
post #240

Earlier quoted context omitted.

The problem arises when non-state actors wreak havoc with drones and AI with inpunity. What would we do if a drone made by a no-name manyfacturer dropped some bombs in Times Square? Who would we blame when someone uses AI to actually sow social mistrust and subvert our existing systems?

> The problem arises when non-state actors wreak havoc with drones and AI with inpunity. Well, for the rest of the world, state actors wreaking havoc with drones and AI with inpunity is already a problem.

truth. how we collectively became accepting of drones used by our governments to destroy targets half way round the world whilst the pilots sit in some skyscraper somewhere in our own countries is remarkable. honestly the disconnect is beyond deeply troubling.

Re: Superintelligence: The Idea That Eats Smart People

#415

Earlier quoted context omitted.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

Which limit are you talking about? I guess you have forgotten what the soldiers of the third reich did.

Or American death squads in Afghanistan. Might as well call them rape and death squads.

What about Guantanamo?

Srebrenica?

Should I go on?

How unenlightened and naive are you?

Re: Superintelligence: The Idea That Eats Smart People

#416

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're dead.

The only thing which keeps you alive is the unwillingness of your neighbors and those who surround you to kill you. The law might punish them afterward, but extensive research has shown that it provides no disuasion to people who are actually willing to kill someone.

A military AI being used to wipe out large numbers of people is exactly as 'inevitable' as the weapons we already have being used to wipe out large numbers of people. The exact same people will be making the decisions and setting the goals. In that scenario, the AI is nothing but a fancy new gun, and I don't see any reason to think it would be used differently in most cases. With drones we have seen the CIA, a civilian intelligence agency, waging war on other nations without any legal basis, but that's primarily a political issue and the fact that it can be done in pure cowardice, without risking the life of those pulling the trigger, which I think is a distinct problem from AI.

Re: Superintelligence: The Idea That Eats Smart People

#417

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

They don't give back? Microsoft employs 100k+ people. That's giving back. These people all pay taxes and give back to society because Microsoft gave them a job. Because Gates happened. And in the case of Gates, let's not forget Bill & Melinda Gates Foundation.

What the hell have you done for society?

This story is very similar for most billionaires. They create A LOT of jobs and careers.

Re: Superintelligence: The Idea That Eats Smart People

#418

He is completely skipping the most important part. The superintelligence has to have some reason to be in conflict with us. Human beings don't go out of their way to hunt down and eliminate ants. They don't find out what ants eat and sieze control of it to manipulate them. There is no reason to think that a superintelligent machine would be likely to present terrible interference to us proposed. So it's super smart a…

>So it's super smart and has its own goals. We can reliably presume that it will need energy to achieve those goals. Will it need to achieve them quickly? Why? Would the superintelligence be shortsighted enough to provoke humans into active combat against it?

It happened so many times throughout human history, that it's really foolish to reject. People with their own goals burned, raped, murdered. Maybe superintelligence will build its own civilization, and to suppress remorse will put people in reservations.

Re: Superintelligence: The Idea That Eats Smart People

#419

What's interesting is the evolution of consciousness. In the long term, does it matter whether or not it's embodied in meat?

I distinguish between consciousness and self-consciousness, but as to your question, I don't think it matters at all in what form self consciousness takes place, and the moral thing to do when something appears to be self-conscious is to treat it with the same kindness you would expect for yourself (unless it is obvious that this something is a clear and present danger). I can totally see how from a moral perspective a human should be put on trial and imprisoned for killing a sentient "machine" - though unfortunately, looking at how as a society we treat animals and many times look the other way at mass cruelty to our own kind, I'm not really sure I'd mind a super-intelligence (and perhaps as a result also benevolent) ruling us instead of the other way around.

Of course, there's the possibility that a super-intelligence would not be able to break through the human egocentric limitations that may have been built into it by its human creators, and in that case, we're screwed...

Re: Superintelligence: The Idea That Eats Smart People

#420
I'm not sure if this is meant to be funny somehow, or taken seriously, but the series of strawmans as arguments, eg. referencing some american sitcom as an example to a phenomen is pretty weak. It is sad, because I can agree with many ideas, but the flawed reasoning is not convincing for someone critical, and wastes those arguments...

Although the read is ovarall interresting, but I find the style and arguments sub par, too american for my taste.

Post reply on HN