Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

141–150 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#141

Post by kiba »

apsec112 wrote: post #88
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Countries that have no rich people are never prosperous. You can raise marginal income tax rates from, say, 60% to 70%, and maybe that's a good idea overall, but it doesn't get rid of billionaires. High-tax Sweden has as many billionaires per capita as the US does: https://en.wikipedia.org/wiki/List_of_Swedes_by_net_worth If you raise the marginal tax rate to 99%, then you get rid of billionaires, but you also kill y…

I don't think it's possible to have a country without rich people, relatively speaking.

In every country we seen have some sort of power hierarchy, therefore unequal distribution of wealth.

Re: Superintelligence: The Idea That Eats Smart People

#142

Post by alexbecker »

panic wrote: post #124
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

It's a military AI that correctly interprets a command to kill a particular group of people, so effectively that its masters start thinking about the next group, and the next. This has been done many times by human-run militaries; would AI make it worse somehow? Groups of humans acting collectively can look a lot like an "AI" from the right perspective. Corporations focused on optimizing their profit spend a huge amo…

I don't think AI will cause a paradigm shift here; but like most powerful technologies, I imagine it will have potent military applications.

Re: Superintelligence: The Idea That Eats Smart People

#143

Post by Analemma_ »

modeless wrote: post #74
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

That's exactly what Maciej spends the last third of the article saying: that the quasi-religious fretting about superintelligence is causing people to ignore the real harm currently being caused by even the nascent AI technology that we have right now.

Re: Superintelligence: The Idea That Eats Smart People

#144

Post by k8t »

argonaut wrote: post #96
k8t wrote: post #72

Let's take a step back and look at the meta-level. So there's maybe a chance AI is extremely dangerous (as in wipes humanity out), and there's a chance that AI might not be dangerous. There are arguments between how likely each choice is. But the more important fact is that since there's so much at stake (all of humanity), we should likely be really really really sure about things. For example, let's say you live in…

"If there's a 1% chance that Pakistani scientists are helping al-Qaeda build or develop a nuclear weapon, we have to treat it as a certainty in terms of our response. It's not about our analysis ... It's about our response." - Dick Cheney

I'm completely oblivious to the al-qaeda situation so I don't really know the context of this. But if I try to get at your logic, it would be like saying:

"See? I just went on a drive and didn't get into a crash. All those people that told me to wear a seat-belt were wrong!"

Re: Superintelligence: The Idea That Eats Smart People

#145

Post by gbhn »

singlow wrote: post #131
zevyoura wrote: post #123

Earlier quoted context omitted.

Don't the people who control military technology in our current time already have "practically unlimited power over everyone else"?

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

True, true, but somehow that's never been much of an obstacle to totalitarian governments. Somehow there's always a soldier who will push the button.

In Nuremberg we developed a way to think about this: people outside the central circles of power have a tremendous amount of pressures they are considering. It's definitely the case that some are sadistic and horrible, but more are just following orders and trying to get by as best they can, and punishing them for war crimes is not appropriate.

The other side of that coin is that it isn't realistic to expect soldiery en masse to resist illegal orders. It's always more complicated than that.

Re: Superintelligence: The Idea That Eats Smart People

#146

Post by Galaxeblaffer »

jimmcslim wrote: post #36
md224 wrote: post #27

"Put another way, this is the premise that the mind arises out of ordinary physics... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise to accept." The thing that irks me about this is how it reinforces a common (and in my opinion, false) dichotomy: either you believe the mind is explicable in terms of ordinary physics or you believe…

Any recommendations on decent pop-sci books that explore the latest understanding of the origin and mechanism of consciousness? Is anyone exploring this third way at all?

I will probably be down voted for this.. But I enjoyed Kurzweil's "How to create a mind" and "on intelligence" by Jeff Hawkins . Even though these books aren't super scientific i really like how these guys think about consciousness and intelligence.

Re: Superintelligence: The Idea That Eats Smart People

#147

Post by seiferteric »

stcredzero wrote: post #32

In 1945, as American physicists were preparing to test the atomic bomb, it occurred to someone to ask if such a test could set the atmosphere on fire...Los Alamos physicists performed the analysis and decided there was a satisfactory margin of safety. Exactly how big was this margin? Please someone tell me this was quite a large margin!

I was really surprised when I read the quote, they were worried about a runaway fusion reaction! I had heard this story before but always thought they were worried about "burning" nitrogen in the chemical sense, ie. creating nitrogen oxides. It seems doubly ridiculous to worry about fusion when we can't even get a fusion reactor to work.

Re: Superintelligence: The Idea That Eats Smart People

#148

Post by openasocket »

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

I'll push back against the idea of smart factories leading to "a vast chasm between a new, tiny Hyperclass and the destitute masses." I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. Income inequality obviously benefits the rich (in that they by definition have more money), but only up to a point. We won't devolve into an aristocracy, at least not because of automation.

Re: Superintelligence: The Idea That Eats Smart People

#149

Post by eltoozero »

The quintessential "rouge AI" scenario, pre-terminator SkyNet but post "Metropolis" is "Colossus: The Forbin Project".

It's finally available on widescreen blu-ray in Germany but Universal has still not re-released it for the western audience.

The filmed exterior of Colossus HQ is Lawrence Hall of Science in Oakland, CA.

Re: Superintelligence: The Idea That Eats Smart People

#150

Post by to3m »

21 wrote: post #85
idlewords wrote: post #56

Earlier quoted context omitted.

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

Of course, if you take your average city dweller on your island, he will probably die of thirst before the tigers get to him. But take an (unarmed) navy seal as your human on the island and I'm pretty sure in a couple of months he will be the Tiger King. And Hawking would just ask his assistant to put the cat into the box. You are artificially depriving him of his actual resources to make a weak point.

What percentage of people are Navy SEALs?

If I were a tiger, I'd probably think this island sounded like an excellent place for a fun adventure holiday with my tiger friends, and I'd be right to do so. Most likely outcome: good food, some exercise, and we can take some monkey heads home for our tiger cubs to play with.

Post reply on HN