Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Superintelligence: The Idea That Eats Smart People (2016)

idlewords.com

Post reply on HN
Print view View original

81–90 of 253 posts

Re: Superintelligence: The Idea That Eats Smart People (2016)

#81

Post by cevn »

ACCount37 wrote: post #71
everyone wrote: post #31

Any of this kinda talk these days is just part of the hype train for the LLM companies tulipomania pyramid scheme. Afaik no-one that is actually working on AGI is anywhere close atm.

Even in the LLM dept: LLMs are the most general AI systems to date, and the performance only ever goes up. Whether adding +5% per model release is enough to get a broadly superhuman system remains to be seen. But my take is that there's no such thing as "not working on AGI" in the frontier labs. Everything that's being put into modern frontier systems is AGI groundwork, one way or the other.

What's 1.05 x 0? Even if you run it 1000 times, it's still 0.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#82

Post by sonofhans »

stoneman24 wrote: post #67
foobiekr wrote: post #38

Earlier quoted context omitted.

There's also the story "Go, Go, Go, Said the Bird" by Sonya Dorman in the first Dangerous Visions which is like this, in a way.

There’s also “the ruum” by Arthur Porges[0]. We got as part of English class in high school, a long time ago. Try not to read the Wikipedia as it might spoil the short story, there’s the pdf available on the web somewhere [0] https://en.wikipedia.org/wiki/The_Ruum

Yes! I remember reading that, as you say, a long time ago. This is the first time I’ve seen someone else reference it. I love that story.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#83

Post by ACCount37 »

cevn wrote: post #81
ACCount37 wrote: post #71

Earlier quoted context omitted.

Even in the LLM dept: LLMs are the most general AI systems to date, and the performance only ever goes up. Whether adding +5% per model release is enough to get a broadly superhuman system remains to be seen. But my take is that there's no such thing as "not working on AGI" in the frontier labs. Everything that's being put into modern frontier systems is AGI groundwork, one way or the other.

What's 1.05 x 0? Even if you run it 1000 times, it's still 0.

Are you trying to say that the capability of modern LLMs is zero?

Because if so, I'm pretty sure any frontier LLM is better at evaluating AI capabilities than you are.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#84

Post by Morromist »

loeg wrote: post #80
andai wrote: post #14

Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human? It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified…

The thing about fiction is, it's fiction. The author can write whatever outcome they want. It doesn't necessarily tell us anything about the real world.

I don't see anything in SOMA that's implausable, so I don't see how it fails to tell us anything about the real world, any more than any other prediction about the future. And we pretty much have to make predictions, its both in our nature and a smart thing to do.

Its possible AI and computing may never be able to reach that level of capability, but we can't know that. One thing that's great about SOMA is that the AI isn't nessessarily very capable and that's part of the problem, its very powerful but its not doing a good job with its enormous task.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#85

Post by dweinus »

andai wrote: post #23

>Is superintelligence just a memetic hazard? [Overblown fear by smart people who are too easily convinced.] Well we can do the wager. If it's a nothingburger, then the worst case scenario is that we approached AI too cautiously. (Ha. What are the odds of that?) If it's not a nothingburger, then we all die, unless the whole world agrees on the correct course of action in advance and coordinates perfectly. Hmm. Well, m…

Ah yes, Pascal's mugging.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#86

Post by PaulHoule »

WithinReason wrote: post #75

I can't say I disagree: We need better scifi! And like so many things, we already have the technology. This is Stanislaw Lem, the great Polish scifi author. English-language scifi is terrible, but in the Eastern bloc we have the goods, and we need to make sure it's exported properly. It's already been translated well into English, it just needs to be better distributed. What sets authors like Lem and the Strugatsky b…

Lem’s Cyberiad was my favorite as a kid and still is!

Re: Superintelligence: The Idea That Eats Smart People (2016)

#87

Post by efskap »

braden-lk wrote: post #41
andai wrote: post #14

Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human? It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified…

One of my favorite games, and I recommend it to anyone who loves a good existential sci-fi horror. If you are not comfortable with stealth games, it also has a "Safe Mode" where enemies aren't a threat to you if you just want to experience the story. Spoiler warning for those that havent played-- I forget the details exactly, but one scene stuck with me. It was a screen in one of the labs, where an experiment was run…

Reminds me of the SCP-like article / short story about the first executable snapshot of a human brain, with similar horrors of realization.

https://qntm.org/mmacevedo

Re: Superintelligence: The Idea That Eats Smart People (2016)

#88

Post by OkayPhysicist »

andai wrote: post #23

>Is superintelligence just a memetic hazard? [Overblown fear by smart people who are too easily convinced.] Well we can do the wager. If it's a nothingburger, then the worst case scenario is that we approached AI too cautiously. (Ha. What are the odds of that?) If it's not a nothingburger, then we all die, unless the whole world agrees on the correct course of action in advance and coordinates perfectly. Hmm. Well, m…

That's just Pascal's wager with a secular/cyberpunk reskin, and has all the same trappings. Why aren't you a monk?

Re: Superintelligence: The Idea That Eats Smart People (2016)

#89

Post by drfloyd51 »

loeg wrote: post #80
andai wrote: post #14

Has anyone played SOMA? Spoiler warning. It explores this idea of, what if there's an AI in charge of ensuring mankind survives at all costs. What would it be willing to do, to keep us alive? Would we even recognize the result as human? It's a horror game and it explores all kinds of fascinating and disturbing scenarios. Simulations of human minds. Artificial worlds. Human minds in robot bodies. Genetically modified…

The thing about fiction is, it's fiction. The author can write whatever outcome they want. It doesn't necessarily tell us anything about the real world.

But it does give us an idea to chew over. And we can determine how “real world” that idea is. Some of us can even see how to bend the idea a little and make a reasonable version of it “real”.

Re: Superintelligence: The Idea That Eats Smart People (2016)

#90

Post by pixl97 »

holmesworcester wrote: post #79

This was one of the rare critiques of AI doom that actually understands the case for it and presents them well, so I kept reading to see what its arguments for our safety against AI doom were. They were roughly: 1. It's hard to put a cat in a box despite us being smarter than a cat, so we're safe. (Counter: we're pretty good at putting cats in boxes when it matters.) 2. It was hard for Australia to kill Emus, so we'r…

1. Like climate change, right?

2. At what cost? Much like the climate change above, you'll have people on the AI side even when it's out in the field extincting us.

4. Adding, over time synthetic data and its generating algorithms can become unaligned with human needs/behaviors (an example would be our current stock market, numbers must go up!).

8. Going back to climate change, it was predicted a long time ago, and while the explosion of automobiles has greatly improved human lives the risks of climate change could erase a lot of that. Might have been better if we dealt with the problem before we have to give the thermometer worried looks.

Post reply on HN