Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

51–60 of 253 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#51

Post by the_af »

andai wrote: post #14

Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human? It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified…

I just love this game. So many good things about it. Not only the one about AI you mentioned, also the issue of consciousness and the self. I simply love how the game tackles this in a way that can only be done with an interactive first person PoV.

Also, I just love this phrase:

> "I woke up in my bed today... a hundred years ago."

Re: Superintelligence: The Idea That Eats Smart People (2016)

#52

Post by blaze33 »

shagie wrote: post #27
forinti wrote: post #15

Earlier quoted context omitted.

There's a story about some Kenyans outrunning a Cheetah in 6km. It had been killing their livestock, so they decided to go after it. Cheetahs are very fast, but humans have way more endurance.

> Cheetahs are very fast, but humans have way more endurance. https://en.wikipedia.org/wiki/Man_versus_Horse_Marathon > The Man versus Horse Marathon is an annual race over 21 miles (34 km), where runners compete against riders on horseback through a mix of road, trail and mountainous terrain. The race, which is a shorter distance than an official marathon road race, takes place in the Welsh town of Llanwrtyd Wells e…

The Man vs Horse marathon is interesting but in a real race we have no actual chance of winning.

There are horse endurance races where the winner arrived in 7,5 hours after 160km[1]. That's a sub 2-hours marathon almost 4 times in a row (not to mention with a guy on your back).

[1] https://eatnstays.com/uaes-almazrouei-wins-almutadil-cup-at-...

Re: Superintelligence: The Idea That Eats Smart People (2016)

#53

Post by stared »

I like this essay, and have very often referred to it when someone talks about AGI. There is a common narrative bias to look at AGI as the Abrahamic God, if not explicitly, then just by saying that it is omniscient, omnipotent, immortal - and will judge us for our deeds.

It is tempting for anyone raised in the West, and immersed in Judeo-Christian culture. And for anyone, in general, as it offers an epic narration of a personal entity.

Yet, the reality might be messier - IMHO closer to biology than to a weird mixture of computer science and theology. There is no ultimate intelligence (see Karpathy’s starfish shapes), just a collection of adaptability, learning, generalization and self-reference. Also, even an extremely smart being (or process) can be fragile.

So, less God, more WAU from SOMA or the Ocean from Solaris.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#54

Post by ctoth »

cute_boi wrote: post #12
jelder wrote: post #2

> If you encountered a cheetah in pre-industrial times (and survived the meeting), you might think it was impossible for anything to go faster. Fun fact, there is no historical evidence of an adult human ever dying from a cheetah attack. They are naturally shy, and a lot smaller than you may realize.

Yep. That said, unlike cheetahs, there’s plenty of evidence of leopards attacking humans. And these days, it’s the leopards, the closed-AI types and misanthropes -- telling everyone, “AI will take your job and there’s nothing you can do about it.”

I keep seeing this and I want to speak to it.

When Dario and others say things like "this is happening and we should probably figure out what to do about it" what ends up happening is people hear "this is happening," see that the person warning them is the person doing the thing, and then short-circuit. "Why can't you just stop then?"

Dario's point, and the point of the people actually trying to solve the problem, is that AI is not just Anthropic and OpenAI. It's the knowledge that you can put more compute in, and get more capability out.

It is a technology now. It exists, in the world. Wishing will not make it go away. Being angry at it will not make it go away. Lying about how much water it uses will not make it go away. If Anthropic and OpenAI Shut down tomorrow, Accenture will not say "oh guess that llm thing won't work, let's go back to hiring humans!"

It is a truth that you can multiply matrices and get something that is economically useful. We cannot un-know this.

Physics allows it, so it will happen. So we should probably figure out what the heck to do about it. If your answer is something along the lines of "restrict it" then 1. let me know how that goes when other people don't, and 2. I really would rather prefer a world where we have the machines do the work the machines can do, not a world where we have human makework. If this means we need to figure out redistribution, let's talk about redistribution!

Re: Superintelligence: The Idea That Eats Smart People (2016)

#55

Post by stared »

andai wrote: post #14

Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human? It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified…

SOMA is existentialism 2.0 (wrote about it here https://p.migdal.pl/blog/2019/06/games-in-which-you-walk-and...).

To concepts you menton, I would add grey goo and x-risk.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#56

Post by dmos62 »

throwuxiytayq wrote: post #36
dmos62 wrote: post #25

> The AI is all-powerful and gives you what you ask for, but interprets everything in a super-literal way that you end up regretting. I like imagining similar discourse when a more basic tool was invented: "A hammer is like a genie, it's all powerful, but, when you hit something with it, it interprets that super-literally, and it hits it."

Isn't this a misinterpretation of what everyone in the AI safety space is worried about, though? I think the idea is that having an AI that interprets everything in a super-literal way would probably be catastrophic, but we can't even build that . It would be a nice world-ending problem to have.

It very well could be, I don't really follow those discussions. Honestly, if I were worried about something on Earth intellectually evolving at a suboptimal pace, it would be humans.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#57

Post by stephencanon »

blaze33 wrote: post #52
shagie wrote: post #27

Earlier quoted context omitted.

> Cheetahs are very fast, but humans have way more endurance. https://en.wikipedia.org/wiki/Man_versus_Horse_Marathon > The Man versus Horse Marathon is an annual race over 21 miles (34 km), where runners compete against riders on horseback through a mix of road, trail and mountainous terrain. The race, which is a shorter distance than an official marathon road race, takes place in the Welsh town of Llanwrtyd Wells e…

The Man vs Horse marathon is interesting but in a real race we have no actual chance of winning. There are horse endurance races where the winner arrived in 7,5 hours after 160km[1]. That's a sub 2-hours marathon almost 4 times in a row (not to mention with a guy on your back). [1] https://eatnstays.com/uaes-almazrouei-wins-almutadil-cup-at-...

“in a real race we have no actual chance of winning” is an absolutely wild thing to say in response to a link to a real race in which the human has won the last few years in a row.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#58

Post by ACCount37 »

Shalomboy wrote: post #33
andai wrote: post #14

Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human? It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified…

SOMA was a cognito-hazard for me and my roommate in college; we played it in the dark together while on some sort of mild hallucinogen and when it came time for Simon to find that high-pressure dive suit, we lost our minds (no pun intended). Watching the WAU twist its way through PATHOS II in whatever way worked first is a particularly jarring analogy for what has happened to our own profession. I can't help but thin…

There's the upcoming ONTOS game that looks like Frictional's SOMA successor.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#59

Post by somesortofthing »

ctoth wrote: post #54
cute_boi wrote: post #12

Earlier quoted context omitted.

Yep. That said, unlike cheetahs, there’s plenty of evidence of leopards attacking humans. And these days, it’s the leopards, the closed-AI types and misanthropes -- telling everyone, “AI will take your job and there’s nothing you can do about it.”

I keep seeing this and I want to speak to it. When Dario and others say things like "this is happening and we should probably figure out what to do about it" what ends up happening is people hear "this is happening," see that the person warning them is the person doing the thing, and then short-circuit. "Why can't you just stop then?" Dario's point, and the point of the people actually trying to solve the problem, is…

A US-China AGI ban treaty could prevent superintelligence indefinitely. Data centers are hard to hide. Have fun buying GPUs when you're cut off from all global payments. America would have to make some unpleasant concessions but that seems like a solid trade for preventing a wide variety of nightmare futures.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#60

Post by willis936 »

yetihehe wrote: post #46
willis936 wrote: post #5

Some of the themes remind me of themes mentioned in this matrix analysis. Specifically I am reminded of the Dune concept of control: "you control what you can destroy" and then asking "do you control your refrigerator?". Sure, you can turn it off but then your food would rot and you might starve. So in a real sense humans have not controlled machines for a long time but have been co evolving in symbiosis. Sure, it's…

I can destroy my refrigerator pretty easily. If I care about food inside, I can take it out into a new one. So, seems I control it by that definition. Your idea "but food will spoil"/"I will lose a small amount of wealth" seems irrelevant to the strict definition. Conversely I think it's a bad definition, it's a show of what is the frame of the mind of the person who states that: "I want to show my control by destroy…

The fridge is a toy example. Take all machinery man has made and destroy it. Will you survive? Or live in a space ship and destroy the ship. Will you survive? Or have a pacemaker or an iron lung or dialysis. How about the simple concept of a combustion engine or any necessary subcomponent, where removing that today would grind all logistics to a halt. How long do you survive? Just because you have one machine that is replaceable doesn't refute that you need machines as much as they need you.

If you so readily dismiss Herbert's definition of control posit a competitor and we can pressure test it. Also, "correlated with a toddler's world view" is not the epic rhetorical refute you think it is.

Post reply on HN