Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People

idlewords.com

Post reply on HN
Print view View original

221–230 of 607 posts

Re: Superintelligence: The Idea That Eats Smart People

#221

Post by [deleted] »

21 wrote: post #155
to3m wrote: post #150

Earlier quoted context omitted.

What percentage of people are Navy SEALs? If I were a tiger, I'd probably think this island sounded like an excellent place for a fun adventure holiday with my tiger friends, and I'd be right to do so. Most likely outcome: good food, some exercise, and we can take some monkey heads home for our tiger cubs to play with.

Fun, until the navy seal appears. It takes only one. The same thing with AI, it takes only one, no matter how many dumber ones we entertained ourselves with before.

[deleted]

Re: Superintelligence: The Idea That Eats Smart People

#222

Post by vkreso »

nradov wrote: post #173
singlow wrote: post #131

Earlier quoted context omitted.

Their might is asymmetrical, but power is mitigated by the willingness of an organization of humans to follow commands. There is a limit to how far a soldier will go, ethically.

Really? https://en.wikipedia.org/wiki/Nanking_Massacre

As horrific as Nanking Massacre was, I believe that doesn't really prove your point. I'd argue that a lot of counter examples of soldiers ethical behaviour simply aren't visible and are forgotten and lost to history (https://en.wikipedia.org/wiki/Survivorship_bias)

Re: Superintelligence: The Idea That Eats Smart People

#223

Post by trishume »

idlewords wrote: post #18
jimmcslim wrote: post #12

'In particular, there's no physical law that puts a cap on intelligence at the level of human beings.' Maybe not, but there are definitely very physical laws governing everything else, that a superintelligent being's ambitions would run into. A superintelligent being isn't going to be able to build a superliminal reactionless drive if the laws of the universe say it isn't possible. More relevant, a superintelligent b…

The place where AI alarmists seem to forget this most often is in computational complexity, and particularly in the power of super-hyper-computers to simulate reality. Bostrom in particular doesn't seem to appreciate the constraints complexity theory puts on what an AI could calculate (unless he thinks P=NP).

The thing is it doesn't need a huge leap in any specific thing, it needs a quantum leap in anything at all. Complexity only constrains you from solving all problems of a given type in reasonable time, not from solving any instance.

There are even many things that we already suspect are just engineering challenges. Like rocketry, hacking and robotics. Even if an AI can just act like a million engineers and scientists working for a decade per second, without any higher level of thinking, it could do some incredible things.

Re: Superintelligence: The Idea That Eats Smart People

#224

Post by breuleux »

openasocket wrote: post #187
myrmidon wrote: post #162

Earlier quoted context omitted.

> I mean, if the masses are destitute, they can't afford the stuff being made at those fancy factories, so the owners of those factories won't make money. I think that's beside the point. Why would the "aristocracy" owning the factories care about money, when they have all the goods (or can trade for them with other aristos)? It's not like they need money to pay other people (the destitute masses are useless to them)…

> Why would the "aristocracy" owning the factories care about money, when they have all the goods (or can trade for them with other aristos)? In this scenario, they only need enough factories to meet the needs of themselves and the other aristocrats. Which means there will be massive demand for goods for the common people, which means there will be jobs for people. The only way your scenario works is if the aristocra…

> In this scenario, they only need enough factories to meet the needs of themselves and the other aristocrats.

That could still end up taking all of the planet's resources, if enough aristocrats try to maximize their military capacity to protect themselves or play some kind of planet-wide game of Risk, or if they create new goods that require absolutely insane amounts of resources.

I mean, ultimately, the issue is that the common people have nothing of value to trade except for resources that they already own, and no way to accumulate things of value. Because of this, the aristocrats will never have an incentive to sell anything to them: if they had any use for the common people's resources, they would just take them. So I think there are three possible outcomes:

* The aristocrats' needs become more and more sophisticated until they need literally the whole planet to meet them, destroying the common people in the process.

* The aristocrats provide to the common people, no strings attached, out of humanism (best case scenario).

* The aristocrats take what they need and let the common people fend off for themselves, creating a parallel economy. In that case the common people would have jobs, although they would live in constant worry that the aristocrats take more land away from them.

Re: Superintelligence: The Idea That Eats Smart People

#225

Post by tachyonbeam »

kazagistar wrote: post #189

They always miss a critical and subtle assumption: that intelligence scales equal to or faster then the computational complexity of improving that intelligence. This is the one assumption I most skeptical of. In my experience, each time you make a system more clever, you also make it MUCH more complex. Maybe there is not hard limit on intelligence, but maybe each generation of improved intelligence actually takes lon…

I think the issue is that once do manage to build an AI that matches human capabilities in every domain, it will be trivial to exceed human capabilities. Logic gates can switch millions of times faster than neurons can pulse. The speed of digital signal also means that artificial brains won't be size-limited by signal latency in the same way that human brains are. We will be able to scale them up, optimize the hardware, make them faster, give them more memory, perfect recall.

Nick Bolstrom keeps going on in his book about the singularity, and about how once AI can improve itself it will quickly be way beyond us. I think the truth is that the AI doesn't need to be self-improving at all to vastly exceed human capabilities. If we can build an AI as smart as we are, then we can probably build a thousand times as smart too.

Re: Superintelligence: The Idea That Eats Smart People

#226

Post by Chathamization »

aaachilless wrote: post #100
idlewords wrote: post #56

Earlier quoted context omitted.

I'm not arguing that it's useless to outsmart a cat. I'm disputing the assumption that being vastly smarter means your opponent is hopelessly outmatched and at your mercy. If you're the first human on an island full of tigers, you're not going to end up as the Tiger King.

What if you're the first human on an island full of baby tigers? I think most AI alarmists would argue that this analogy is vastly more appropriate.

I think an analogy of a baby human on an island of tigers would be vastly more appropriate. Humans would be like the tigers - they might have a lower overall intelligence, but they are mature and self sufficient.

A hyper-intelligent AI would be more akin to a baby human - it must be taught and raised first. But like the baby human on the island, it wouldn't be able to be taught by its peers, or benefit from the generations that came before it. It would certainly be at the mercy of the tigers until it matured, and even after it matured we wouldn't expect it to be able to use language or bows, or be anything other than a wild man. It probably wouldn't even seem much more intelligent than the tigers on the island.

Re: Superintelligence: The Idea That Eats Smart People

#227

Post by tbt »

idlewords wrote: post #62
mrfusion wrote: post #28

I've always wondered if the problems an intelligence solves are exponentially hard so even if we build a super intelligence it wouldn't be all that much smarter than we are. For example compare how many more cities in the traveling salesman problem a super computer can solve vs your grandmas pc. It's more but surprisingly not all that many more. What do you think of that idea?

I think this basic concept of intractability, which programmers are very familiar with, hasn't penetrated far enough into AI world. Bostrom and Yudkowsky in particular seem happy to hand-wave past computational complexity.

Have you seen gwern's essay on computational complexity arguments about AI? https://www.gwern.net/Complexity%20vs%20AI

Re: Superintelligence: The Idea That Eats Smart People

#228

Post by trishume »

narrator wrote: post #167
AnimalMuppet wrote: post #108

Earlier quoted context omitted.

I don't buy the AI quasi-religious stuff. But your argument here is flawed. If protein folding can do the process in constant time, we may be able to find another process (but electronic rather than wet chem) that can also do it in constant time.

Being able to find constant time algorithms for algorithms that currently take exponential time is not at all assured.

It is to some extent if we have a constant time example in real life. If the AI can't solve protein folding fast enough it can just design absurdly fast protein sequencers and really good microscopes and get proteins to fold themselves in real life and use the results in the rest of the computation.

Re: Superintelligence: The Idea That Eats Smart People

#229

Post by Klathmon »

ebbv wrote: post #152
apsec112 wrote: post #53

"I live in California, which has the highest poverty rate in the United States, even though it's home to Silicon Valley. I see my rich industry doing nothing to improve the lives of everyday people and indigent people around us." This is trivially false. Over a hundred billionaires have now pledged to donate the majority of their wealth, and the list includes many tech people like Bill Gates, Larry Ellison, Mark Zuck…

Err, I hate to be the one to break it to you but those billionaires pledging to donate their wealth? It's just a tax dodge. They're moving their money into foundations before we pass stronger tax laws than we have at the moment. And it allows their families to continue to live off of (via salaries for running the foundations) the wealth for generations to come. Which is not to say they don't do some good work with it…

Personally I'd rather give billionaires more reasons donate their money than less.

If setting up a foundation that actually helps a ton of people means that their families can get a fraction of that money back, that's fine with me...

Unless you are implying that they are getting more money back through the salaries than they are in donating. In which case i'd love to see a source.

Re: Superintelligence: The Idea That Eats Smart People

#230

Post by Analemma_ »

taneq wrote: post #217
Analemma_ wrote: post #39

Earlier quoted context omitted.

> The problem with argument ad hominem isn't that it never works. It often leads to the correct conclusion ... You immediately self-contradicted here. If a decision heuristic often leads to the correct conclusion, then it's a good heuristic (if you are under time constraints, which we are). Of course, given unlimited time to think about it, we would never use ad hominem reasoning and consider each and every argument…

A good heuristic leads to a reduction in the overall cost of a decision (combining the cost of making the decision with the cost of the consequences if you get it wrong). A heuristic like "it's risky to rent a car to a male under 25" saves a lot of cost in terms of making the decision (background checks, accurately assessing the potential renter's driving skills and attitude towards safety, etc.) and has minimal down…

I agree with you in principle, but the combination of the base rate for "26-year-olds redefining reality" being so low and the consequences being not nearly as dire as you make out mean I stand by my claim, at least for the case of heuristics on how to identify dangerous cults.

With regards to the Einstein bit, per my above comment I still think that skepticism of GR was perfectly rational right up until it got an empirical demonstration. And it's not like the consequences for disbelieving Einstein prior to 1919 were that dire: the people who embraced relativity before then didn't see any major benefit for doing so, nor did it hurt society all that much (there was no technology between 1915 and 1919 that could've taken advantage of it).

Post reply on HN