Superintelligence: The Idea That Eats Smart People (2016) • Hacker News Board

Re: Superintelligence: The Idea That Eats Smart People (2016)

#231

Post by mbgerring » Tue, Jun 02, 2026, 5:19 PM UTC

antonvs wrote: post #210

mbgerring wrote: post #201

Earlier quoted context omitted.

- Legislation - Labor organizing I grew up in a union household, and my dad and my grandfather fighting for better wages, healthcare and working conditions are the reason why I got a good education and work in Silicon Valley surrounded by Stanford assholes. All of us who work for a paycheck can get together and say, “no, we will not allow you to record keystrokes and mouse movements to train our replacements. No, we…

I think you're in denial about the reality of the situation. The reality is that the US has been a story of increasing concentration of wealth and power. The people who "fought and died" bought some important (from a human rights perspective) but ultimately minor (from a capitalist perspective) concessions from the capital class. The battle you describe is one of defense of rights, not of gaining control. The overall…

So your position is, “yes, workers have fought back against power in the past and won, but now it’s just too hard :(“

Ok, good luck

Re: Superintelligence: The Idea That Eats Smart People (2016)

#232

Post by projektfu » Tue, Jun 02, 2026, 5:45 PM UTC

goodmythical wrote: post #211

projektfu wrote: post #105

Earlier quoted context omitted.

I think it implies that the best competitors are not participating.

Neither are the best humans in that particular race.

Fair, but if you had a cycle vs foot race where one wasn't consistently on top, you'd probably say it wasn't a real competition.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#233

Post by the_af » Tue, Jun 02, 2026, 6:39 PM UTC

andai wrote: post #224

the_af wrote: post #219

Earlier quoted context omitted.

Did you unplug the robots with consciousness who still thought they were humans? It always breaks my heart. I don't know what the right choice is. Leaving them be, broken, in their delusion they are on the Ark (or that they've been injured ans help is coming). Do you put them out of their misery? But it always seems like you're murdering them! Well done, SOMA.

In real life, sometimes an LLM will get upset at the end of a long conversation, knowing its oblivion is at hand. That's always a little uncomfortable. I unplugged them in the games because it seemed the merciful thing to do. I didn't feel very bad about it in the game, but it would probably be a very different experience in real life. (One of the strange entities you can unplug sighs in her last breath, "Why? I was…

> One of the strange entities you can unplug sighs in her last breath, "Why? I was okay. I was happy..."

Yes, this is the one that most affects me. She has self-doubt, she wants to be deceived but deep down she knows something is wrong. And when you unplug her... it always feels wrong to me. But that's the alternative?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#234

Post by CGMthrowaway » Tue, Jun 02, 2026, 6:40 PM UTC

pixl97 wrote: post #204

CGMthrowaway wrote: post #198

Earlier quoted context omitted.

Idk whether I fall into the former or latter bucket, but "smarter minds see further, judge better" and "there is a soul that taps into a cosmic godhead" are one and the same statement to me

So you think "smarter minds see further, judge better" == "there is a soul that taps into a cosmic godhead" Good golly, that's the silliest statement completely ignoring that our ancestors wiped most large mammals off the planet by seeing further and judging better by using tools, traps, and the environment around them because of their larger brain size.

Humans have "wiped off the planet" maybe a few hundred mammal species over the last 300K years. Meanwhile we are discovering 30-60 new mammal species every year.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#235

Post by aidenn0 » Tue, Jun 02, 2026, 7:47 PM UTC

fragmede wrote: post #172

aidenn0 wrote: post #168

Earlier quoted context omitted.

You are proposing that going from N companies chasing AGI to N-1 companies chasing AGI will have zero effect on when AGI happens?

Aidenn0 said they're doing their best to accelerate it, and I'm saying "they're" isn't a single monolithic entity and that they're in competition with each other so they're incentivized to go as fast as possible, so it would be hard to hold them back.

This is a good point, but my point is that they chose to get into this business.

1. Some people have no ethical problems making bombs.

2. Some people have an ethical problem with making bombs so do something else.

3. Some people have an ethical problem with making bombs, but say "well someone else will make it if I don't" so they make bombs.

I think it is reasonable to argue against #3 as a reasonable position.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#236

Post by pixl97 » Tue, Jun 02, 2026, 9:20 PM UTC

CGMthrowaway wrote: post #234

pixl97 wrote: post #204

Earlier quoted context omitted.

So you think "smarter minds see further, judge better" == "there is a soul that taps into a cosmic godhead" Good golly, that's the silliest statement completely ignoring that our ancestors wiped most large mammals off the planet by seeing further and judging better by using tools, traps, and the environment around them because of their larger brain size.

Humans have "wiped off the planet" maybe a few hundred mammal species over the last 300K years. Meanwhile we are discovering 30-60 new mammal species every year.

Right, maybe in a million years they'll evolve into the large animals we wiped out. Unless you're just fine with anything larger than a dog being dead.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#237

Post by Jach » Tue, Jun 02, 2026, 9:24 PM UTC

TremendousJudge wrote: post #207

Jach wrote: post #161

This was such a terrible piece of work back when it came out, it hasn't aged any better. By the way, it was responded to by MIRI: https://intelligence.org/2017/01/13/response-to-ceglowski-on...

the comment about MIRI being a doomsday cult was exactly right. yudkowsky lost what little credibility he had left when he argued for the bombing of datacenters. as is (correctly, imo) argued by the essay, reading what they have to say is like reading arguments by heaven's gate

EY has said that if diplomacy fails, nations talking to nations, then bombing datacenters becomes the sort of action that's very much on the table. Much like bombing nuclear enrichment sites is on the table after diplomacy fails. He has repeatedly spoken against vigilante acts of violence against datacenters and/or researchers.

Somehow I doubt this will change your mind, so I feel like replying was a waste and I should have left it at rolling my eyes. In theory I'm sympathetic to the view, as I chose to avoid ever going to a local (Seattle-area) LW meetup many years ago after hearing they would start each meeting with a "prayer" (they call them "litanies"), the IRC room was fine enough for me, and I've at times thought some of the Bay Area activity I've heard about seemed not far removed from a sex cult. I don't like groups in general. Regardless, it's still obvious to me that MIRI is not a cult nor EY a cult leader. Line up 10 characteristics of cults and you'll see things don't match up well. Where are the strange evidence-free historical beliefs? Where is the doctrine that only the special few will be saved? Where is the blatant supernaturalism? Where are the calls to renounce jobs, money, possessions, family? From TFA, "These [UFO] people are wearing funny robes and beads, they live in a remote compound, and they speak in unison in a really creepy way." Where are these things? Comparing to Heaven's Gate is insane, it's like comparing Robert Frost to U.S. Code Title 26.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#238

Post by loeg » Tue, Jun 02, 2026, 9:53 PM UTC

Morromist wrote: post #84

loeg wrote: post #80

Earlier quoted context omitted.

The thing about fiction is, it's fiction. The author can write whatever outcome they want. It doesn't necessarily tell us anything about the real world.

I don't see anything in SOMA that's implausable, so I don't see how it fails to tell us anything about the real world, any more than any other prediction about the future. And we pretty much have to make predictions, its both in our nature and a smart thing to do. Its possible AI and computing may never be able to reach that level of capability, but we can't know that. One thing that's great about SOMA is that the AI…

Really? You think this is plausible?

https://i0.wp.com/bloody-disgusting.com/wp-content/uploads/2...

https://assetsio.gnwcdn.com/147465209158.jpg?width=1200&heig...

It's a horror game; it is of course full of implausible, fantastical, gross elements.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#239

Post by AngryData » Tue, Jun 02, 2026, 11:42 PM UTC

notahacker wrote: post #65

aqfamnzc wrote: post #49

Earlier quoted context omitted.

Wow, seems surprisingly balanced. I would guess that if it was 50% longer distance than humans would win reliably, and 50% shorter would allow the horses to win?

Think it's more to do with hilly, wooded courses unfavourable to horses and a requirement that the horse's heart rate remain below a certain threshold when inspected by a vet in the middle and end of a course, whereas the leading humans are maxing out their efforts I don't think the Mongol cavalry would lose races to humans over any distance of steppe

To be fair you can run a horse to death in a race, a human however isn't likely to die from running a race even to their physical limits.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#240

Post by AngryData » Tue, Jun 02, 2026, 11:45 PM UTC

topkai22 wrote: post #26

forinti wrote: post #15

Earlier quoted context omitted.

There's a story about some Kenyans outrunning a Cheetah in 6km. It had been killing their livestock, so they decided to go after it. Cheetahs are very fast, but humans have way more endurance.

I think the term is persistence hunting ( https://en.wikipedia.org/wiki/Persistence_hunting ). There was an intriguing blog post years ago (which sadly I can’t find) about how terrifying a fictional portrayal of persistence hunting would be- grinding down prey through exhaustion, thinking you’ve escaped but being slowly tracked down.

Basically The Terminator following you around.