Earlier quoted context omitted.
If a human can apply AI irresponsibly, the AI can probably also learn to apply AI irresponsibly.
Not really, humans are the ones who think and have agency.
Do you have any source to back that claim up?
There is no hard takeoff
31–40 of 190 posts
Re: There is no hard takeoff
#32He makes a good point. The only real way to "summon the demon" is if AGI is centralized. If each of us has a maximally intelligent AGI in our phone - and it is us as individuals controlling the prompts, then the playing field is levelled. Unstoppable force meets unstoppable force. Gun meets gun. A tale as old as society.
Re: There is no hard takeoff
#33> The problem is your model needs to include all the computers playing the market, and it also needs to include the other hedge fund bros themselves.
The error here is equating hard takeoff with such granular and expensive world prediction. Human intelligence finds more efficient compression than this. When Donald Trump decides what provocative thing to say in a speech, he isn't granularly modeling the human intelligence of his supporters, opponents, journalists, and media organizations in a particularly CPU-costly way. He's just discovered some narrower domain where there's a vein of predictability that simplifies the whole system enough that he can guess a tweet or speech will be provocative in the desired way. The same thing happens when a scientist is studying cold fusion or dangerous viruses.
You don't have to predict what will work, you just have to predict what might work, and try and try again. AI doesn't have to be good at predicting the future in any absolute terms. It just has to be better than us at predicting what is worth trying.
Once it is better than us at deciding what's worth trying, why not FOOM? FOOM isn't guaranteed perhaps, but why is it not at least one of the likely outcomes?
Re: There is no hard takeoff
#34This article reminds me a bit about a core theme in Isaac Asimov's Foundation series, which I'm coincidentally reading (mild spoiler warning). The science of psychohistory that is used to predict the future and create the Seldon Plan, only works if the future is allowed to unfold uninformed by psychohistory itself. That's why the Foundation was founded without any psychologists or knowledge of the Seldon plan. Seldon's predictions are based on the behaviour of all human of the galaxy, but knowledge of those predictions changes the outcomes, so they are no longer reliable.
As soon as AI starts participating in our world, the data that it was trained on from before AI was participating is no longer representative of the status quo. The very presence of AI changes the world it operates in.
Re: There is no hard takeoff
#35> Back in 2014, Elon Musk referred to AI as summoning the demon. And it wasn’t hard to see that view. Soon, Go agents would beat top humans learning from self play. By the end of 2017, the same algorithm mastered Chess and Shogi. By 2020, it didn’t even need tons of calls to the simulator, and could play Atari too. > AI looked scary. It looked like it was one FOOM away from self playing and becoming superhuman at the…
> And yet, here we are in 2023 and self driving cars still don’t work.
Cruise and Waymo were today given licenses to operate across San Francisco 24/7.
Self driving cars do work and are here. They just need LiDAR.
Re: There is no hard takeoff
#36This article reminds me a bit about a core theme in Isaac Asimov's Foundation series, which I'm coincidentally reading (mild spoiler warning). The science of psychohistory that is used to predict the future and create the Seldon Plan, only works if the future is allowed to unfold uninformed by psychohistory itself. That's why the Foundation was founded without any psychologists or knowledge of the Seldon plan. Seldon…
Reminds me a bit of how mathematics gets into all sorts of trouble once self-reference / recursion is introduced.
Re: There is no hard takeoff
#37Earlier quoted context omitted.
Not really, humans are the ones who think and have agency.
Do you have any source to back that claim up?
Just look up any serious, non-quacky overview about how it works. If that’s excessive, just consider the empirical fact that we have created it (that’s also why we know exactly how it works), that might be enough.
Re: There is no hard takeoff
#38I will be worried when AI will constantly beat top1 league of legends team in normal environment
Chess is way simpler imo.
Re: There is no hard takeoff
#39[deleted]
Re: There is no hard takeoff
#40Am I the only one who sees a jumbled stream of barely coherent semi-thoughts and word salad?