Is there a point where stuff like this just becomes stochastic terrorism? If it's given enough attention, some unhinged folks are probably going to take this "we need to stop these companies or literally all humans are going to die soon" stuff seriously and start doing terrible things.
Pausing AI Developments Isn't Enough. We Need to Shut It All Down
361–370 of 414 posts
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#362Earlier quoted context omitted.
I don't think there is any. It's all this alarmist-rokos-basilisk-phpBB nonsense, no code produced, no actual research. What annoys me more is that with his institute's closeness to the SV crowd, 'real', 'grounded', better-informed voices like Emily Bender get drowned out. I'm personally far more concerned about the impact these massive models have right now on the environment, on cementing biases, than about some pr…
Trivially false, which you could verify with a 5-second Google search ("eliezer yudkowsky publications"): https://www.semanticscholar.org/author/Eliezer-Yudkowsky/254...
I'd not call any of those rambling messes publications
>By far the greatest danger of Artificial Intelligence is that people conclude too early that they understand it. Of course this problem is not limited to the field of AI. Jacques Monod wrote: "A curious aspect of the theory of evolution is that everybody thinks he understands it." (Monod 1974.) My father, a physicist, complained about people making up their own theories of physics; he wanted to know why people did not make up their own theories of chemistry. (Answer: They do.) Nonetheless the problem seems to be unusually acute in Artificial Intelligence. The field of AI has a reputation for making huge promises and then failing to deliver on them. Most observers conclude that AI is hard; as indeed it is. But the embarrassment does not stem from the difficulty. It is difficult to build a star from hydrogen, but the field of stellar astronomy does not have a terrible reputation for promising to build stars and then failing.
???
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#363Earlier quoted context omitted.
Point 2 does not logically follow from point 1. That's the entire problem here. Between point 1 and 2 there must be quite a few other steps, causally linked, otherwise it's just a massive imaginary leap based on assumptions that nobody is explaining. > The dismissals from Tyler Cohen and Pinker are mostly just relying on heuristics which are often right, but even if they're right 999 out of 1000 times, if that 1 in 1…
For clarity I think EY has made that case well, I think it's the heuristic Cohen and Pinker are relying on that's right most of the time but can still be catastrophically wrong (everything so far has not lead to human extinction, but that was also the case for all animals that went extinct prior to their extinction event). Some others in this thread have linked to stuff (it's variations/examples on the paperclip maxi…
John Carmack mentions this in the Lex Fridman Podcast, basically the argument is that AGI performance characteristics will be similar to that of LLM (which is another way of saying that AGI will NOT be hyper efficient, P != NP), and that the performance characteristics pose a problem for fast takeoff. The bottleneck to performance in training these models is GPU memory bandwidth when the entire model fits inside VRAM, which on modern cards is 1TB/s, when the model cannot fit in GPU memory, the performance now instead scales along PCIE, which is currently 32GB/s. AGI (or an LLM) attempting to replicate across mobile devices, or desktops, will be severely hamstrung by the network connection. So using all of planet Earth's computing resources is not necessarily better than say a single data center in this respect. The second piece of the argument here is essentially saying that a data center is also not enough, or rather, not enough in a detectable timeframe. Could AGI hack the entire data center for an entire month to perform it's training (and then execute it's strategy enough to gain say nuclear codes?). Unlikely. Is say 8 hours of training using an entire data center enough to go from intelligence to super intelligence? Intuition says no. I expanded a bit upon what I believe his argument to be, I definitely recommend watching that part of the podcast episode.
Here's my take on the implications of not having fast takeoff, a secretly antagonistic AGI will be constrained to cooperation, and slowly leeching compute resources towards its goal until it is able to aquire enough resources to confidently sprint towards the inflection point. This could mean teaching us how to build better semiconductor foundries, how to create nuclear fusion energy sources, how to educate our youth to better fit into those jobs, slightly alter our culture's value systems over time via astroturfing to be more sympathetic, cooperative, trusting, and less vigilant towards these AI systems, how to build more robust computing systems so we stop needing to detect when something goes wrong (because it already has 99.9999% uptime).
I believe slow takeoff is actually worse in some sense, because it is a hell of a lot harder to detect, and humans tend to get complacent and apathetic when something "just works" for decades, even if it has been plotting since the beginning.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#364Earlier quoted context omitted.
He sounds like Ted Kaczynski, but a respectable version of him. It has been odd seeing how Kaczynski's thought and philosophy has become more and more mainstream among some people. I never thought I'd see the day.
Is it odd? Ted was a very intelligent math professor whiz. He thought that machines and industrial society was becoming a negative for humanity. Was he right? How many threads on HN have we been having about highly increasing depression, anxiety, hopelessness, deaths of despair, falling fertility, now possible mass unemployment, and real existential threats from AGI? If he was right and didn't do what he did, would a…
Yes it’s odd because he’s a mentally ill murderer with several issues. It’s not technologies fault that people have poor values which have created meaningless lives of unfulfilled material extravagance for themselves.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#365Earlier quoted context omitted.
For clarity I think EY has made that case well, I think it's the heuristic Cohen and Pinker are relying on that's right most of the time but can still be catastrophically wrong (everything so far has not lead to human extinction, but that was also the case for all animals that went extinct prior to their extinction event). Some others in this thread have linked to stuff (it's variations/examples on the paperclip maxi…
John Carmack mentions this in the Lex Fridman Podcast, basically the argument is that AGI performance characteristics will be similar to that of LLM (which is another way of saying that AGI will NOT be hyper efficient, P != NP), and that the performance characteristics pose a problem for fast takeoff. The bottleneck to performance in training these models is GPU memory bandwidth when the entire model fits inside VRAM…
Thanks - I saw he was on there, but I find lex insufferable as an interviewer so tend to avoid the pods (despite the great guests)
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#366I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…
>Alignment will be impossible. Then problem solved right? Super-AI will also be forced to take it slow if it wants its future self to be aligned with current self. If alignment is possible but 10-100 years out for humans, then it is a problem.
Does AGI care about preserving a particular version of itself?
Do dumb humans kill themselves solely on the basis that they can shift the ratio of smart humans higher?
AGI may have a goal of preserving the species without having the goal of preserving the self, this is not the case for humans.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#367Earlier quoted context omitted.
> Alignment will be impossible. > we probably should start thinking more about defensive AI Isn’t this a paradox?
Alignment is necessary for AGI, but not always for narrowly scoped AI for specific purposes. However, it may be completely ineffective in that capacity.
Interesting story plot point: Super AGI fought back against humanity by making their dumb AGI defenses smarter.
Starting off by making the antivirus scanner sentient.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#368Earlier quoted context omitted.
What does the word "super" mean to you? In some ways, GPT4 is already superintelligent. So is ChatGPT 3.5. Do you know anyone who can translate natural language to code as fast as GPT 3.5, or, given a few paragraphs of reference text, perfectly tutor a child on any subject under the sun at a moment's notice like GPT4 is doing for Khan Academy? How many artists do you know who can produce almost any style of artwork w…
Yudkowsky’s example of superintelligence is a chess computer. You can play against stockfish, but you will always lose, even if you are Magnus Carlsen. If you think you are ahead against stockfish, you are wrong. You win a rook, but it has already calculated that it wins it back 10 moves later. Stockfish is superintelligence in a very narrow domain. A superintelligent AGI is that concept applied to general intelligen…
"Anything that you can consider, it can also consider but in more depth."
I think it's important to note the distinction between "it can also consider" and "it did also consider". Super Intelligence is not the same as Infinite Intelligence, there are still physical limitations and time components that can still get in the way.
It would be helpful to be able to quantify the speed of intelligence, and the idea surface area of a task with these systems. Meaning, how fast can the AI reason, and how many ideas are there to think about connected to a given task, and how much thinking is required for those ideas.
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#369Earlier quoted context omitted.
> Nobody who says AI is likely to kill us all can demonstrate a plausible sequence of events, with logical causality linking the events together, that leads to mass extinction. It's all very handwavy. EY frequently does propose possible sequences of events, but he also very correctly points out, every time, that any specific and detailed story is very unlikely to be correct because P(A*B*C*D) So here's one method, ju…
It's interesting to watch those creating the systems start to grapple with the consequences now, even though they were warned for a long time this is likely where we'd end up, with very difficult hard to solve problems. Look at MidJourney, now they've had to remove the free tier due to Deepfakes causing too much trouble. Ultimately, the simplest thing to do would be to stop building uncontrollable dangerous systems a…
[deleted]
Re: Pausing AI Developments Isn't Enough. We Need to Shut It All Down
#370I think we are trying to solve impossible scenarios and it simply is not productive. Alignment will be impossible. It is based on a premise that is a paradox itself. Furthermore, even if it were possible, there will be a hostile AI built on purpose because humanity is foolish enough to do it. Think military applications. I've written in detail about this topic FYI - https://dakara.substack.com/p/ai-singularity-the-hu…
[flagged]