Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

91–100 of 253 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#91

Post by CamperBob2 »

stared wrote: post #55
andai wrote: post #14

Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human? It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified…

SOMA is existentialism 2.0 (wrote about it here https://p.migdal.pl/blog/2019/06/games-in-which-you-walk-and... ). To concepts you menton, I would add grey goo and x-risk.

Neat. Not sure if your site is a gold mine inside a rabbit hole, or a rabbit hole inside a gold mine, but I really dig both the aesthetic and the content.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#92

Post by kbrkbr »

smaudet wrote: post #64
stared wrote: post #53

I like this essay, and have very often referred to it when someone talks about AGI. There is a common narrative bias to look at AGI as the Abrahamic God, if not explicitly, then just by saying that it is omniscient, omnipotent, immortal - and will judge us for our deeds. It is tempting for anyone raised in the West, and immersed in Judeo-Christian culture. And for anyone, in general, as it offers an epic narration of…

For all its talk of inoculation, this is a terribly written essay. They do not make a point, nor even arguments, instead, opting to ramble in hopes that you forget whatever it was you were thinking. The issue is simple. Just like us (who are arguably complex, look at what we're building over here, this AI computer stuff!), entities have simple core needs (like food, water, power, etc.). An infinitely smart AGI has th…

From my point of view we have simply no idea what a infinitely smart AGI is and how to build it.

How would it make the combinatorial explosion in state space search go away, to pick one example?

And if it doesn't, is it then an infinitely smart AGI?

The concept seems to assume all problems humans struggle with can be solved. The halting problem is one witness that this is probably not true.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#93

Post by tptacek »

topkai22 wrote: post #26
forinti wrote: post #15

Earlier quoted context omitted.

There's a story about some Kenyans outrunning a Cheetah in 6km. It had been killing their livestock, so they decided to go after it. Cheetahs are very fast, but humans have way more endurance.

I think the term is persistence hunting ( https://en.wikipedia.org/wiki/Persistence_hunting ). There was an intriguing blog post years ago (which sadly I can’t find) about how terrifying a fictional portrayal of persistence hunting would be- grinding down prey through exhaustion, thinking you’ve escaped but being slowly tracked down.

https://en.wikipedia.org/wiki/It_Follows

Re: Superintelligence: The Idea That Eats Smart People (2016)

#94

Post by emtel »

The "Einstein can't get a cat into a carrier even though he is smarter" is just a hilariously bad argument. All cat owners can get their cats into a carrier! And most cats don't want to get in, because they hate the vet! And it's almost entirely because the humans are smarter!

You can even get a literal tiger into a carrier, even though it can kill you easily. You just drug its food and wait till it passes out. This is because you are smarter than it, and know that tranquilizers exist and how to obtain them, which is a strategy that cats of any size are not even able to conceive of, and probably can't understand what happened after it's been done to them.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#95

Post by famouswaffles »

emtel wrote: post #94

The "Einstein can't get a cat into a carrier even though he is smarter" is just a hilariously bad argument. All cat owners can get their cats into a carrier! And most cats don't want to get in, because they hate the vet! And it's almost entirely because the humans are smarter! You can even get a literal tiger into a carrier, even though it can kill you easily. You just drug its food and wait till it passes out. This…

Yeah the whole thing is strange. He does such a good job in the first bit outlining all the reasons for caution that I was intrigued to see what he had to say against it, but it's just one bad argument after another. Even Hawking's cat is bad. Make some money and pay someone to get the cat in the box.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#96

Post by ethin »

emtel wrote: post #94

The "Einstein can't get a cat into a carrier even though he is smarter" is just a hilariously bad argument. All cat owners can get their cats into a carrier! And most cats don't want to get in, because they hate the vet! And it's almost entirely because the humans are smarter! You can even get a literal tiger into a carrier, even though it can kill you easily. You just drug its food and wait till it passes out. This…

I... Think you completely missed the point, which is that each and every method you enumerated is a brute-force tactic.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#97

Post by suzzer99 »

emtel wrote: post #94

The "Einstein can't get a cat into a carrier even though he is smarter" is just a hilariously bad argument. All cat owners can get their cats into a carrier! And most cats don't want to get in, because they hate the vet! And it's almost entirely because the humans are smarter! You can even get a literal tiger into a carrier, even though it can kill you easily. You just drug its food and wait till it passes out. This…

> You just drug its food and wait till it passes out.

Human zoo keepers are actually smarter than that. For months, they train the tiger to go into the carrier to get food. Then on transport day, they shut the door behind it. Unclear if this works for future transport situations.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#99

Post by suzzer99 »

jelder wrote: post #2

> If you encountered a cheetah in pre-industrial times (and survived the meeting), you might think it was impossible for anything to go faster. Fun fact, there is no historical evidence of an adult human ever dying from a cheetah attack. They are naturally shy, and a lot smaller than you may realize.

Ice yachts get well over 100 mph. I'm not sure how much they were used in pre-industrial times.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#100

Post by garaetjjte »

ACCount37 wrote: post #70
blamestross wrote: post #6

AI Superintelligence doesn't scare me for the same reasons "grey goo" doesn't scare me. We are awash in self-replicating machines. The biosphere is already a grey-goo apocalypse. Any new competitors have a serious moat to cross to out compete any existing self-replicators. We are awash in intelligent agents. Our society (and meta society) is full of superhuman agents already. There is a huge moat for any new intellig…

"Intelligent agents" we have around run on a metabolic budget of 25W and a hardware platform the size of a melon. Human intelligence doesn't scale upwards well. Individual humans only get this smart, and there are gains from getting multiple humans to work together - but the more of them you add, the larger is your communication and coordination overhead. In no small part because humans are self-interested agents tha…

>the more of them you add, the larger is your communication and coordination overhead. In no small part because humans are self-interested agents that simply aren't designed to compose their capabilities seamlessly.

What proves that AI doesn't have the same limitations? There's only so much computation you can do in given space, and all communication is limited by universal speed limit.

Post reply on HN