Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

451–460 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#451

Post by TeMPOraL »

jmagoon wrote: post #338
TeMPOraL wrote: post #274

Earlier quoted context omitted.

There's also another thing. AI may not need to be superhuman, it may be close-but-not-quite human and yet be more effective than us - simply because we carry a huge baggage of stuff that a mind we build won't have. Trust me, if I were to be wired directly to the Internet and had some well-defined goals, I'd be much more effective at it than any of us here - possibly any of us here combined. Because as a human, I have…

What about consciousness or intelligence implies that it would be 'pure' in the sense that you describe? Wouldn't a fully conscious being have a great deal of complexity that might render it equivalent to the roommate example? Couldn't it get offended after crawling the internet and reading that a lot of people didn't like it very much? The idea that 'intelligence' is somehow an isolatable and trainable property igno…

It doesn't ignore those examples. The idea pretty much comes from the definition of intelligence used in AI, which (while still messy at times) is more precise than common usage of the world.

In particular, intelligence is a powerful optimization process - it's agent's ability to figure out how to make the world it lives in look more like it wants. Values on the other hand, describe what the agent wants. Hence the orthogonality thesis, which is pretty obvious from this definition. 'idlewords touches on it, but only to try and bash it in a pretty dumb way - the argument is essentially like saying "2D space doesn't exist, because the only piece of paper I saw ever had two dots on it, and those dots were on top of each other".

You could argue that for evolution the orthogonality thesis doesn't hold - that maybe our intelligence is intertwined with our values. But that's because evolution is a dynamic system (a very stupid dynamic system). Thus it doesn't get to explore the whole phase space[0] at will, but follows a trajectory through it. It may be so that all trajectories starting from the initial conditions on our planet end up tightly grouped around human-like intelligence and values. But not being able to randomize your way "out there" doesn't make the phase space itself disappear, nor does it imply that it is inaccessible for us now.

--

[0] - https://en.wikipedia.org/wiki/Phase_space

Re: Superintelligence: The Idea That Eats Smart People

#452

Post by tripzilch »

state_less wrote: post #154
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

The internet also brought us wikipedia, google, machine learning and a place to talk about the internet. Machine learning advances are predicated on the internet, will grow the internet and will become what we already ought to know we are. A globe spanning hyper intelligence working to make more intelligence at break neck pace. Somewhere along this accelerating continuum of intelligence, we need to consciously decide…

which part of the "we need better scifi" slide did you not understand?

Re: Superintelligence: The Idea That Eats Smart People

#453

Post by bryanrasmussen »

I think these are problems if indeed it turns out that intelligence correlates with goal making and pursuing of said goals, which given the other article on the front page at the moment (that I am totally intelligent enough to link to but also unmotivated to do so) which says that intelligence is not linked to success I just don't know that we can assume that correlation.

Re: Superintelligence: The Idea That Eats Smart People

#454

Post by tripzilch »

hueving wrote: post #236
pmyjavec wrote: post #77

Earlier quoted context omitted.

I feel the problem is the fact there is 100 billionaires in he first place, no one gets rich on their own. Gates et al, are clever, but didn't get where they are totally independently without others support, so they should give back. Also, some of these billionaires are running companies that are great at tax avoidance, probably most of them. Now what? They get to pick and choose where they get to spend there/invest…

Can you explain to me why you think taking all of the wealth of 100 billionaires will help the poor? 100 billion spread over the population of California is less than $3000/person. So you can wipe out all of the billionaires and give everyone $50/week for one year, what is that going to change?

> you can wipe out all of the billionaires and give everyone $50/week for one year, what is that going to change?

well if you put it like that, the world ...

Re: Superintelligence: The Idea That Eats Smart People

#455

Post by drawkbox »

Klathmon wrote: post #229
ebbv wrote: post #152

Earlier quoted context omitted.

Err, I hate to be the one to break it to you but those billionaires pledging to donate their wealth? It's just a tax dodge. They're moving their money into foundations before we pass stronger tax laws than we have at the moment. And it allows their families to continue to live off of (via salaries for running the foundations) the wealth for generations to come. Which is not to say they don't do some good work with it…

Personally I'd rather give billionaires more reasons donate their money than less. If setting up a foundation that actually helps a ton of people means that their families can get a fraction of that money back, that's fine with me... Unless you are implying that they are getting more money back through the salaries than they are in donating. In which case i'd love to see a source.

Similar reason that CEOs take the $1 salary, purely stock growth mixed with lower taxes and more under the radar to lower their taxable income [1]

Most rich people that are mega wealthy pay minimal tax by taking loans against their foundations/trusts. As you know there are no taxes on loans but a percentage, that may or may not just go right back to them. They essentially use the trusts/loans and the fact that they are essentially zero risk to take out other loans that get interest rates below inflation, sometimes very minimal interest[2]. That is essentially free money. Should the interest rates ever adjust if not fixed they can buy outright quickly.

The same thing happens with trusts/foundations. They have this hoarded money sitting there as a base they can leverage. Yes it is nice some will go to others when they pass but ultimately they would not do this if they could not use this leverage advantage. It is essentially a hack on lowering or even completely removing any taxable income as long as you have it.

[1] http://www.cnbc.com/id/46236916

[2] http://www.wsj.com/articles/SB100014241278873235270045790791...

Re: Superintelligence: The Idea That Eats Smart People

#456

Post by joefourier »

nickbauman wrote: post #83

Moore's Law (on a single CPU core) effectively ended around 2009. Our programming idioms for parallel processing are still so crude, hardly anyone can write maintainable parallelized code for anything but toy problems. Until we can restart Moore's Law again, this is all academic. http://www.globalnerdy.com/2007/09/07/multicore-musings/

Toy problems? GPGPU is widely used in real-world applications such as image processing, compression, computer graphics, physical simulations, cryptography and machine learning, which are all embarrassingly parallel. With regards to programming, I would not all CUDA and OpenCL crude at all.

Re: Superintelligence: The Idea That Eats Smart People

#457

Post by mtgx »

alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Yes, that's the ultimate threat. But in the meantime, the threat is the military will think the AI is "good enough" to start killing on its own and the AI actually gets it wrong a lot of the time.

Kind of like what we're already seeing now in Courts, and kind of how NSA and CIA's own algorithms for assigning a target are still far less than 99% accurate.

Re: Superintelligence: The Idea That Eats Smart People

#458

Post by mtgx »

modeless wrote: post #74
alexbecker wrote: post #51

While I agree with Maciej's central point, I think the inside arguments he presents are pretty weak. I think that AI risk is not a pressing concern even if you grant the AI risk crowd's assumptions. Elided from https://alexcbecker.net/blog.html#against-ai-risk : The real AI risk isn't an all-powerful savant which misinterprets a command to "make everyone on Earth happy" and destroys the Earth. It's a military AI that…

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

This is basically why a lot of people didn't want Google to become a defense contractor while researching military robots. If it did, it would've naturally started to use DeepMind for it. And that's a scary thought.

Re: Superintelligence: The Idea That Eats Smart People

#459

Post by mtgx »

otakucode wrote: post #416
modeless wrote: post #74

Earlier quoted context omitted.

Exactly. The "Terminator" scenario of a rogue malfunctioning AI is a silly distraction from the real AI threat, which is military AIs that don't malfunction. They will give their human masters practically unlimited power over everyone else. And AI is not the only technology with the potential to worsen inequality in the world.

Human beings have been extremely easy to kill for our entire existence. No system of laws can possibly keep you alive if your neighbors are willing to kill you, and nothing can make them actually unable to kill you. Your neighbor could walk over and put a blade in your jugular, you're dead. They could drive into you at 15MPH with their car, you're dead. They could set your house on fire while you're asleep, you're de…

This is exactly the same argument of targeted surveillance vs mass surveillance.

Saying "humans have spied on each other for centuries" is nothing but a distraction for how far beyond what should be legal mass surveillance is, because it makes it so easy to have everything on everyone all the time. It's nothing like pass targeted surveillance. If anything it's much more like Stasi-style surveillance, but on a much bigger and more indepth scale. And we already know how dangerous such surveillance is with the wrong person leading a country.

When war becomes easy and cheap (to the attacker), you'll just end up having more of it. It doesn't help the the military industrial complex constantly lobbies for it either.

Re: Superintelligence: The Idea That Eats Smart People

#460

Post by ccozan »

kabdib wrote: post #445
Senji wrote: post #384

Earlier quoted context omitted.

You can always use LEO EMP nukes to bring us back to the stone age, thus taking out the AI.

You don't even need to do that; just stop mining coal, or operating oil and gas fields, or scram the reactors. Or more easily, open some circuit breakers; in a pinch you can take down a few electrical towers (not many). A rogue AI's "oxygen" is electrical power, which is really kind of fragile. The emergency power for most datacenters won't last more than a few days without replenishing diesel fuel. For a distributed…

What stopps the AI to take the path from Matrix?
Post reply on HN