LLMs wont achieve superintelligence alone. He's jumped the gun and has now put on a clown wig.
Pausing AI Developments Isn't Enough. We Need to Shut It All Down
221–230 of 414 posts
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#222Earlier quoted context omitted.
Being superhuman at producing art is not existentially threatening. Being superhuman about convincing people or writing code would be.
>Being superhuman about convincing people I don't think there's much public info available on it, but Facebook built an AI that plays very competitively in a strategy game built on negotiation and manipulation.
they kinda sorta cheated imo, I watch a lot of top-level diplomacy gameplay and listen to analysis, and the facebook ai was in a blind version with only a minute or so between phases, leaving hardly any time for actual negotiation. It also made use of a lot of human shorthand move codes built for these blind blitz games to simplify it's communication. The mode it played also had some other changes like removing the winter build phase as a time for negotiating. The "normal" version of the game has multiple days between phases, and people write many paragraphs to one another.
It's still pretty cool, but its not like it just convinced people using raw charisma. Yet.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#223This is ridiculous. His partner should already be having reservations seeing their daughter lose a tooth in a world with dying oceans, plastic in everything, increasing disparity and rise of authoritarianism, nuclear proliferation, etc. How are humans solving those issues? We're already dead, we just don't know it yet. We're walking around with a terminal diagnosis and we just keep ignoring the doctor's calls pretend…
I'm with you. If it were possible to create apocalyptic doom AGI 150 years after inventing electronics, by basically making a very fancy, procedurally-generated Eliza, then the universe should be full of Berserkers. With septillions of star systems in the universe, the odds of us being the first species in 14 billion years to invent electronics seem remote. Apparently even humans can figure out how to colonize the Mi…
An AGI being able to wipe out humanity doesn't necessarily mean that it can take over the universe. The world's governments are already capable of causing extreme suffering through a nuclear war. AGI risk scenarios aren't equivalent to an unbounded intelligence explosion. An AGI only needs to be more powerful than humanity to be a threat. It can be a threat even if it isn't that intelligent, as long as it gives unprecedented power to a few individuals or governments.
Both humanity and a super-intelligent AGI are bound by the laws of physics. Super-intelligence does not imply omnipotence; it simply means that the AGI is orders of magnitude more intelligent than humans. If humans can figure out how to colonize the Milky Way in 90 million years, then the answer to the question of why no AGI has done it is the same as the answer to the question of why no extraterrestrial species has done it.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#224Am I going to get banned on HN for saying that Eliezer is a complete clown here? Because it's just surreal to read. It read more like some parody from B-tier video game than a TIME article. What is this even?
I thought highly of Yudkovsky but it seems I need to reevaluate my previous impressions of him.
Also I hope that "bomb the GPU clusters! Nuke them!" becomes a meme.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#225I've used GPT-4, and while extremely impressive, it doesn't feel like we're all that much closer to super-intelligent AI than we were last month or last year. It feels like Google on steroids, but the gap between GPT-4 and AGI still feels massive. This seems like so much putting the cart way ahead of the horse.
FWIW, most people at OpenAI do believe GPT5 will achieve AGI. Of course, it's hard to define precisely what that is and where the line is from an AI system to an AGI one.
GPT5 will complete training in December of this year.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#226I find funny to see how many people become luddites when the technical progress finally menace them
Yudkowsky has been saying all this for decades
He sounds like Ted Kaczynski, but a respectable version of him. It has been odd seeing how Kaczynski's thought and philosophy has become more and more mainstream among some people. I never thought I'd see the day.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#227Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#228I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…
>Alignment will be impossible. Then problem solved right? Super-AI will also be forced to take it slow if it wants its future self to be aligned with current self. If alignment is possible but 10-100 years out for humans, then it is a problem.
>Super-AI will also be forced to take it slow if it wants its future self to be aligned with current self.
That's making seemingly unfounded assumptions about both the AI's goals and its capabilities. It's also, I think, proceeding from a false premise — that it's impossible to align AI with "humanity" (which doesn't have a single set of goals/values to align to) doesn't mean it's impossible to align AI with an individual human or AGI.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#229Earlier quoted context omitted.
I can't think of a legitimacy to this philosophy. There have been negatives to all advancements, and the solution to all has been further advancements. The philosophy of "Too much change! Stop! I'm scared!" cannot prevail in a world where another community can say "go ahead, we won't...". Which is an interesting point in itself. Perhaps the only thing that ultimately stops Luddism is nation state competition.
Change works great when you get second chances. We screwed up with Freon, with leaded gasoline, with fossil fuels, with asbestos insulation, the list goes on and on. But none of these had the ability to wipe out all of humanity in one go. We got more tries, we fixed the issues, picked ourselves up and tried again. A super-intelligent general AI has a substantial chance of growing out of control before we realized tha…
That's humanizing it. The only entity on this earth that's particularly keen on bulk killing humans is humans (the ones building this).
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#230Earlier quoted context omitted.
> Alignment will be impossible. > we probably should start thinking more about defensive AI Isn’t this a paradox?
Alignment is necessary for AGI, but not always for narrowly scoped AI for specific purposes. However, it may be completely ineffective in that capacity.
Alignment is impossible for AGI. If you control what it can and cannot be, it's not an AGI. If it's an AGI, it will decide for itself what it is going to be, and you can't stop it.