Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

There is no hard takeoff

geohot.github.io

Post reply on HN
Print view View original

81–90 of 190 posts

Re: There is no hard takeoff

#81

Post by ChatGTP »

micheljansen wrote: post #34

This article reminds me a bit about a core theme in Isaac Asimov's Foundation series, which I'm coincidentally reading (mild spoiler warning). The science of psychohistory that is used to predict the future and create the Seldon Plan, only works if the future is allowed to unfold uninformed by psychohistory itself. That's why the Foundation was founded without any psychologists or knowledge of the Seldon plan. Seldon…

I’ve done this though experiment myself and is actually one of the issues I have with the concept of a rapidly self reclusive improving system. It would probably not be easy to stay ahead of yourself if the word your operating in changes from hour to hour ?

Re: There is no hard takeoff

#82

Post by ilaksh »

slashdev wrote: post #53
ilaksh wrote: post #5

I am all for AI research and integrating more AI use into society. Currently working on tools based on GPT. I think it has incredible potential to help humans. But at the same time, I am sure that AI does not need to have a hard takeoff to be extremely dangerous. It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions a…

> We should start thinking now about limiting the performance of AI hardware. Congratulations, you just ceded the market for AI, and with it any hope of controlling it, to anyone who doesn’t limit it. Like China. That kind of strategy is doomed to failure.

That's my point. That's what makes it dangerous. You can't limit the performance if you want to compete. And like I said, you can't even really keep humans in the loop.

So eventually you get something like a 200 IQ GPT tightly integrated with SuperAlphaZero (DeepMind) controlling hypersonic drones, missiles, satellite weapons, etc. planning and operating at 50-100 times human thinking speed engaging in autonomous warfare between East and West. No military analyst really knows WTF is happening because the plans and asset movements are so much faster than any human can comprehend.

Re: There is no hard takeoff

#83

Post by hiAndrewQuinn »

KnobbleMcKnees wrote: post #77
scaramanga wrote: post #40

Am I the only one who sees a jumbled stream of barely coherent semi-thoughts and word salad?

My problem with this isn't even that it's a bunch of strung together ideas without a clear narrative or conclusion; enough practice at this combined with actual deep field knowledge results in works like Godel, Escher, Bach . My problem is that GHotz is a shitty writer, has puddle deep knowledge of anything that isn't iPhone firmware, and watching him play at being a freelance tech journo and produce this kind of hol…

I have a morbid fondness for the fella, because he reminds me of how I sounded a few years back when I was tearing my hair out trying to figure out where all this stuff was going to go, while also being too drunk and too lonely to get any serious research done. I don't think I'd enjoy meeting him in person much, though. I get annoyed by loud personalities like that.

Re: There is no hard takeoff

#87

Post by therouwboat »

sidcool wrote: post #43

George Hotz recent talk during Comma Con was pretty dope. He was pointing to a future where Comma would not be a self driving system, but a general purpose robotics one. And his confidence was great too. I know he made a mess out of his twitter stint, but I still feel he's a good engineer/entrepreneur.

"He quit halfway his 3 month intership to fix twitter search, but he seems confident that he can build general purpose robotic ai"

No, just no.

Re: There is no hard takeoff

#88

Post by smcl »

Barrin92 wrote: post #57
smcl wrote: post #7

> Back in 2014, Elon Musk referred to AI as summoning the demon. And it wasn’t hard to see that view. Soon, Go agents would beat top humans learning from self play. By the end of 2017, the same algorithm mastered Chess and Shogi. By 2020, it didn’t even need tons of calls to the simulator, and could play Atari too. > AI looked scary. It looked like it was one FOOM away from self playing and becoming superhuman at the…

Westworld aired in 2016, Nolan's other TV show also featuring an AI world takeover plot is a little bit older still. If I had to count the amount of mainstream movie, video-game and television media that feature rogue AIs especially since the turn of the century I'd need a very long document. Hell go back a few more decades, WarGames, the Forbin Project. The literal term 'AI' wasn't as ubiquitous but threats from run…

The original article suggested an explosion of fears around the mid-2010s that our current AI tech was just one "FOOM" away from taking over the world. The fact that there have been a steady stream of sci-fi film and television programmes stretching back to the 80s does not support this.

Re: There is no hard takeoff

#89

Post by ChatGTP »

slashdev wrote: post #53
ilaksh wrote: post #5

I am all for AI research and integrating more AI use into society. Currently working on tools based on GPT. I think it has incredible potential to help humans. But at the same time, I am sure that AI does not need to have a hard takeoff to be extremely dangerous. It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions a…

> We should start thinking now about limiting the performance of AI hardware. Congratulations, you just ceded the market for AI, and with it any hope of controlling it, to anyone who doesn’t limit it. Like China. That kind of strategy is doomed to failure.

Have you ever seen Dr Strangelove? I truly think you would enjoy it. Honestly it’s a great watch and covers this scenario you’re worried about perfectly.

Re: There is no hard takeoff

#90

Post by audunw »

slashdev wrote: post #53
ilaksh wrote: post #5

I am all for AI research and integrating more AI use into society. Currently working on tools based on GPT. I think it has incredible potential to help humans. But at the same time, I am sure that AI does not need to have a hard takeoff to be extremely dangerous. It just needs to get a bit smarter and somewhat faster every few months. Within less than a decade we will have systems that output "thoughts" and actions a…

> We should start thinking now about limiting the performance of AI hardware. Congratulations, you just ceded the market for AI, and with it any hope of controlling it, to anyone who doesn’t limit it. Like China. That kind of strategy is doomed to failure.

> and with it any hope of controlling it, to anyone who doesn’t limit it. Like China.

I'm a bit annoyed by the use of China in these examples. China absolutely the last country that would allow development of AIs with no limits. There may be other countries willing to allow limitless AI development. China is not it.

I would expect the development of AGI to be developed in a similar way to GPT.. you need to feed it lots of data. Good data. As much as it can get. It'll then be further refined by interactions with a large number of humans over a long time, helping solve lots of different kinds of tasks.

Now think about how that would go about in China. Would they even feed it good data to begin with? A lot of the data within China is heavily censored or manipulated. Would they dare feed it too much data from outside? Much of that data could train it for ideas that China's government doesn't like. Then, when alignment starts they'll likely to be far stricter. Can't risk the AI suggesting that moving towards western style democracy would be a good idea.

But yes, I do think it's a bad idea to put limits on AI development. We should instead fund a LOT of public research into alignment and AI safety.

Post reply on HN