Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

111–120 of 253 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#111

Post by [deleted] »

kbrkbr wrote: post #92
smaudet wrote: post #64

Earlier quoted context omitted.

For all its talk of inoculation, this is a terribly written essay. They do not make a point, nor even arguments, instead, opting to ramble in hopes that you forget whatever it was you were thinking. The issue is simple. Just like us (who are arguably complex, look at what we're building over here, this AI computer stuff!), entities have simple core needs (like food, water, power, etc.). An infinitely smart AGI has th…

From my point of view we have simply no idea what a infinitely smart AGI is and how to build it. How would it make the combinatorial explosion in state space search go away, to pick one example? And if it doesn't, is it then an infinitely smart AGI? The concept seems to assume all problems humans struggle with can be solved. The halting problem is one witness that this is probably not true.

[deleted]

Re: Superintelligence: The Idea That Eats Smart People (2016)

#112

Post by ctoth »

mbgerring wrote: post #63
ctoth wrote: post #54

Earlier quoted context omitted.

I keep seeing this and I want to speak to it. When Dario and others say things like "this is happening and we should probably figure out what to do about it" what ends up happening is people hear "this is happening," see that the person warning them is the person doing the thing, and then short-circuit. "Why can't you just stop then?" Dario's point, and the point of the people actually trying to solve the problem, is…

Physics allows this, and actually taking advantage of it requires billions of dollars of unprecedented infrastructure buildout that is already destabilizing the power grid . The only reason that infrastructure buildout is happening at all is the ideological capture of a small handful of obscenely wealthy people, who are fueling this buildout by spreading the extreme paranoia you’re echoing here. I do not understand w…

> This entire thing is extremely historically contingent, and we could easily stop this train tomorrow.

So, the Baruch Plan?

The Manhattan Project was $~2B in 1945 dollars, and a national-scale industrial mobilization. Now North Korea has the bomb. That's with nuclear material, which doesn't get easier and easier and easier to work with every year.

Compare to the price to train GPT-2 in 2019 ($43,000), and in 2026 ($73) [0].

[0]: https://x.com/karpathy/status/2017703360393318587

Re: Superintelligence: The Idea That Eats Smart People (2016)

#113

Post by soledades »

altruios wrote: post #110
CGMthrowaway wrote: post #106

> Premise 2: The brain is an ordinary configuration of matter, albeit an extraordinarily complicated one. If we knew enough, and had the technology, we could exactly copy its structure and emulate its behavior with electronic components.... The mind arises out of ordinary physics.... If you are very religious, you might believe that a brain is not possible without a soul. But for most of us, this is an easy premise t…

What do even mean by 'soul'? What do you mean by 'supernatural' - and (assuming your definition is the standard one of 'not detectable by any measurement') by what mechanism that could possibly affect physical matter? (the onus is on you to prove the positive claim that there exists the supernatural or soul to begin with, which there is currently no evidence for). The concept is self defeating by its own definition,…

"Matter" is an abstaction. Mind, properly considered, (i.e. not with words) is not.

Strict adherence to Occham's razor would have us dispense with the former, but the latter is useful empirically.

There is some dogmatic insistence in GP, but equally dogmatic throwdowns on the other side of the argument are often passed over as trivially obvious.

I don't know what's what, but I think this insistence is a useful counterweight.

Onuses are on whomever says they exist ;)

Re: Superintelligence: The Idea That Eats Smart People (2016)

#114

Post by FergusArgyll »

shagie wrote: post #27
forinti wrote: post #15

Earlier quoted context omitted.

There's a story about some Kenyans outrunning a Cheetah in 6km. It had been killing their livestock, so they decided to go after it. Cheetahs are very fast, but humans have way more endurance.

> Cheetahs are very fast, but humans have way more endurance. https://en.wikipedia.org/wiki/Man_versus_Horse_Marathon > The Man versus Horse Marathon is an annual race over 21 miles (34 km), where runners compete against riders on horseback through a mix of road, trail and mountainous terrain. The race, which is a shorter distance than an official marathon road race, takes place in the Welsh town of Llanwrtyd Wells e…

There was a great radiolab episode about it a few years ago but I remember it being in Utah not UK

Re: Superintelligence: The Idea That Eats Smart People (2016)

#115

Post by altruios »

soledades wrote: post #113
altruios wrote: post #110

Earlier quoted context omitted.

What do even mean by 'soul'? What do you mean by 'supernatural' - and (assuming your definition is the standard one of 'not detectable by any measurement') by what mechanism that could possibly affect physical matter? (the onus is on you to prove the positive claim that there exists the supernatural or soul to begin with, which there is currently no evidence for). The concept is self defeating by its own definition,…

"Matter" is an abstaction. Mind, properly considered, (i.e. not with words) is not. Strict adherence to Occham's razor would have us dispense with the former, but the latter is useful empirically. There is some dogmatic insistence in GP, but equally dogmatic throwdowns on the other side of the argument are often passed over as trivially obvious. I don't know what's what, but I think this insistence is a useful counte…

> "Matter" is an abstaction. Mind, properly considered, (i.e. not with words) is not.

...wat?

> Strict adherence to Occham's razor would have us dispense with the former, but the latter is useful empirically.

No: you have that reversed. Matter can be reasoned about, matter is a useful abstraction, e=mc^2. energy = matter*speed of light^2. No such formula exists for the mind.

> I don't know what's what, but I think this insistence is a useful counterweight.

Why is insistence a useful counter weight to factual arguments?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#116

Post by wasabi991011 »

yetihehe wrote: post #46
willis936 wrote: post #5

Some of the themes remind me of themes mentioned in this matrix analysis. Specifically I am reminded of the Dune concept of control: "you control what you can destroy" and then asking "do you control your refrigerator?". Sure, you can turn it off but then your food would rot and you might starve. So in a real sense humans have not controlled machines for a long time but have been co evolving in symbiosis. Sure, it's…

I can destroy my refrigerator pretty easily. If I care about food inside, I can take it out into a new one. So, seems I control it by that definition. Your idea "but food will spoil"/"I will lose a small amount of wealth" seems irrelevant to the strict definition. Conversely I think it's a bad definition, it's a show of what is the frame of the mind of the person who states that: "I want to show my control by destroy…

Why do you think it's a bad definition?

Not for fridges, I think that was a bad example. But it seems accurate at the level of geopolitics, where e.g. Iran shows it controls Hormuz by closing it with mines and other weaponry.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#117

Post by tim333 »

Animats wrote: post #9

It's amusing to read people in the past writing about the prospect of superhuman intelligence. The real problems have turned out to be different. Sycophancy and hallucinations, which are part of being confidently wrong, remains a big problem. Needing square miles of data centers was an issue in 1950s science fiction, and disappeared by the 1980s. Yet now they're being built, with private funding and the prospect of p…

We don't really have superhuman intelligence yet, as in way ahead. The current stuff is ahead at some things and bad at others.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#118

Post by andai »

OkayPhysicist wrote: post #88
andai wrote: post #23

>Is superintelligence just a memetic hazard? [Overblown fear by smart people who are too easily convinced.] Well we can do the wager. If it's a nothingburger, then the worst case scenario is that we approached AI too cautiously. (Ha. What are the odds of that?) If it's not a nothingburger, then we all die, unless the whole world agrees on the correct course of action in advance and coordinates perfectly. Hmm. Well, m…

That's just Pascal's wager with a secular/cyberpunk reskin, and has all the same trappings. Why aren't you a monk?

Yeah. Turns out game theory is useful in games.

I don't understand the monk question though.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#119

Post by XorNot »

hilariously wrote: post #68
braden-lk wrote: post #41

Earlier quoted context omitted.

One of my favorite games, and I recommend it to anyone who loves a good existential sci-fi horror. If you are not comfortable with stealth games, it also has a "Safe Mode" where enemies aren't a threat to you if you just want to experience the story. Spoiler warning for those that havent played-- I forget the details exactly, but one scene stuck with me. It was a screen in one of the labs, where an experiment was run…

There's definitely a short story in Accelerando about this same topic.

I mean all of Accelerando is kind of about this: the idea is introduced in chapter 1 when you meet the lobsters.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#120

Post by twosdai »

neaden wrote: post #101
notahacker wrote: post #65

Earlier quoted context omitted.

Think it's more to do with hilly, wooded courses unfavourable to horses and a requirement that the horse's heart rate remain below a certain threshold when inspected by a vet in the middle and end of a course, whereas the leading humans are maxing out their efforts I don't think the Mongol cavalry would lose races to humans over any distance of steppe

Yeah I think on say, a proper road the horse would win at any distance.

There's probably some upper limit to it, i am not sure a horse could live through the ultra marathon moab race in the western us.
Post reply on HN