Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

151–160 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#151

Post by mundo »

kobayashi wrote: post #122

I can't disagree enough. Having recently read Superintelligence, I can say that most of the quotes taken from Bostrom's work were disingenuously cherry-picked to suit this author's argument. S/he did not write in good faith. To build a straw man out of Bostrom's theses completely undercuts the purpose of this counterpoint. If you haven't yet read Superintelligence or this article, turn back now. Read Superintelligenc…

People will stop downvoting this if you edit it to add a specific example or three.

Re: Superintelligence: The Idea That Eats Smart People

#152

Post by ebbv »

apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

Err, I hate to be the one to break it to you but those billionaires pledging to donate their wealth? It's just a tax dodge. They're moving their money into foundations before we pass stronger tax laws than we have at the moment. And it allows their families to continue to live off of (via salaries for running the foundations) the wealth for generations to come.

Which is not to say they don't do some good work with it, the Bill and Melinda Gates foundation has done some great work fighting malaria and bringing fresh water to poor communities.

But these same foundations also do a lot of other work, like furthering charter schools which benefits wealthy families to the detriment of poor communities here in the US.

Re: Superintelligence: The Idea That Eats Smart People

#153

Post by Girlang »

zoren wrote: post #76
apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

As to the first claim. It seems Mississippi has the highest poverty rate at 21.9%. California is at 16.4%. Source: https://en.wikipedia.org/wiki/List_of_U.S._states_by_poverty...

@dang should delete that post. We shouldn't be spreading fake news here.

Re: Superintelligence: The Idea That Eats Smart People

#154

Post by state_less »

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

The internet also brought us wikipedia, google, machine learning and a place to talk about the internet.

Machine learning advances are predicated on the internet, will grow the internet and will become what we already ought to know we are. A globe spanning hyper intelligence working to make more intelligence at break neck pace.

Somewhere along this accelerating continuum of intelligence, we need to consciously decide to make thing awesome. So people aim to build competent self driving cars, that way fewer people die of drunk driving or boredom. Let's keep trying. Keep trying to give without thought of getting something in return. Try to make the world you want to live in. Take a stand against things that are harmful to your body (in the large sense and small sense) and your character. Live long and prosper!!!

Re: Superintelligence: The Idea That Eats Smart People

#155

Post by 21 »

to3m wrote: post #150
21 wrote: post #85

Earlier quoted context omitted.

Of course, if you take your average city dweller on your island, he will probably die of thirst before the tigers get to him. But take an (unarmed) navy seal as your human on the island and I'm pretty sure in a couple of months he will be the Tiger King. And Hawking would just ask his assistant to put the cat into the box. You are artificially depriving him of his actual resources to make a weak point.

What percentage of people are Navy SEALs? If I were a tiger, I'd probably think this island sounded like an excellent place for a fun adventure holiday with my tiger friends, and I'd be right to do so. Most likely outcome: good food, some exercise, and we can take some monkey heads home for our tiger cubs to play with.

Fun, until the navy seal appears. It takes only one.

The same thing with AI, it takes only one, no matter how many dumber ones we entertained ourselves with before.

Re: Superintelligence: The Idea That Eats Smart People

#156

Post by zardo »

singlow wrote: post #131
zevyoura wrote: post #123

Earlier quoted context omitted.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

>There is a limit to how far a soldier will go, ethically.

What horrible things to you have in mind that armies have not already done?

Re: Superintelligence: The Idea That Eats Smart People

#157

Post by jzwinck »

> an artificial intelligence would also initially not be embodied, it would be sitting on a server somewhere, lacking agency in the world. It would have to talk to people to get what it wants.

Organized crime and semi-organized criminal gangs stand to establish a highly effective symbiosis with amoral machines which "lack agency."

If a machine wants to kill someone, all it needs to do is find a person to carry out the task in exchange for some benefit such as cash or blackmail ("I will report you to the police for helping me steal the electricity I need, plus the small trafficking operation I helped you optimize last summer").

Two arms and two legs are not what make modern criminals scary. It is their ability to plan, optimize and repeat sophisticated operations. Soon there will be an app for that.

Re: Superintelligence: The Idea That Eats Smart People

#158

Post by argonaut »

nickbauman wrote: post #83

Moore's Law (on a single CPU core) effectively ended around 2009. Our programming idioms for parallel processing are still so crude, hardly anyone can write maintainable parallelized code for anything but toy problems. Until we can restart Moore's Law again, this is all academic. http://www.globalnerdy.com/2007/09/07/multicore-musings/

To be fair, many machine learning algorithms (especially deep learning) are very parallelizable, both when training and predicting.

Re: Superintelligence: The Idea That Eats Smart People

#159

Post by Afforess »

>Observe that in these scenarios the AIs are evil by default, just like a plant on an alien planet would probably be poisonous by default.

I believe this is a core misunderstanding. Bostrom never says that a superintelligent AI is evil by default. Bostrom argues the AI will be orthogonal, it's goals will be underspecified in such a way that leads it to destroy humanity. The paperclip optimizer AI doesn't want to kill people, it just doesn't notice them, the same way you don't notice ants you drive over in your daily commute. AIs with goals orthogonal to our own will attack humanity in the same way humanity attacks the rainforests, piecemeal, as-needed, and without remorse or care for what was there before. It won't be evil, it will be uncaring, and blind.

Re: Superintelligence: The Idea That Eats Smart People

#160

Post by argonaut »

k8t wrote: post #144
argonaut wrote: post #96

Earlier quoted context omitted.

"If there's a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response. It's not about our analysis ... It's about our response." - Dick Cheney

I'm completely oblivious to the al-qaeda situation so I don't really know the context of this. But if I try to get at your logic, it would be like saying: "See? I just went on a drive and didn't get into a crash. All those people that told me to wear a seat-belt were wrong!"

If you think those two anecdotes are the same, good luck!
Post reply on HN